2465 Sqoop Jobs

Required Skills & Experience-Pharma Domain 5+ years of experience in IT with 3+ years in Big Data and Cloud Data Engineering. Strong expertise in Hadoop ecosystem HDFS, Hive, Sqoop, Spark. Proficiency in PySpark , SQL , and Python for data transformation and automation. Hands-on experience with Databricks notebooks and DBFS . Deep knowledge of AWS Cloud Services (S3, Glue, Redshift, Lambda, DMS, Athena, EC2, IAM). Experience with Azure Data Factory, Azure Synapse, and Azure Data Lake (ADLS) . Familiarity with Snowflake , DBT , and Airflow for pipeline orchestration and transformation. Strong understanding of data quality, performance tuning, and optimization techniques in Spark. Experience w...

Job Summary: We are seeking a skilled and detail-oriented Big Data Engineer with over 5 years of IT experience, including at least 3 years specializing in Big Data application development using the Hadoop ecosystem and Cloud platforms (AWS/Azure) . The ideal candidate will have strong expertise in PySpark , ETL pipeline development , data integration , and workflow orchestration to build scalable and efficient data solutions. Key Responsibilities: Design, develop, and optimize Big Data pipelines using PySpark , Hive , Sqoop , and Spark SQL . Implement ETL processes for ingesting, transforming, and loading large-scale structured and unstructured data from diverse sources (JSON, Parquet, Avro,...

NEX Softsys is looking for Big Data Developer to join our dynamic team and embark on a rewarding career journey Design, develop, and maintain big data solutions to meet business requirements and support data-driven decision making Work with stakeholders to understand their data needs and determine how to best use big data technologies to meet those needs Design and implement scalable, high-performance big data architectures, using technologies such as Hadoop, Spark, and NoSQL databases Extract, transform, and load large data sets into a big data platform for analysis and reporting Write complex SQL queries and develop custom scripts to process big data Collaborate with data scientists, data ...

Job Description: About Us At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. Responsible Growth is how we run our company and how we deliver for our clients, teammates, communities and shareholders every day. One of the keys to driving Responsible Growth is being a great place to work for our teammates around the world. We’re devoted to being a diverse and inclusive workplace for everyone. We hire individuals with a broad range of backgrounds and experiences and invest heavily in our teammates and their families by offering competitive benefits to support their physical, emotional, and financial well-being. Bank of...

Role Description : As a Senior Cloud Data Platform (GCP) Engineer at Incedo, you will be responsible for managing and optimizing the Google Cloud Platform environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or Azure and have experience with big data technologies such as Hadoop or Spark. You will be responsible for configuring and optimizing the GCP environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security tea...

Lead who can manage deliverables from offshore, ensures adequate design/technical support is provided PySpark, Python, Data bricks PySpark, Python/Scala, Data bricks, Understanding on Streaming Pipeline, Docker & Kubernetes Problem solving and technical troubleshooting skills Scala (Good to have) Mandatory Competencies Big Data - Big Data - Pyspark Beh - Communication Data Science and Machine Learning - Data Science and Machine Learning - Databricks Data Science and Machine Learning - Data Science and Machine Learning - Python DevOps/Configuration Mgmt - DevOps/Configuration Mgmt - Containerization (Docker, Kubernetes).

Role Description : As a Senior Cloud Data Platform (GCP) Engineer at Incedo, you will be responsible for managing and optimizing the Google Cloud Platform environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or Azure and have experience with big data technologies such as Hadoop or Spark. You will be responsible for configuring and optimizing the GCP environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security tea...

Location: Gurgaon and Bangalore Experience: 8+ years (data engineering / analytics engineering), with previous lead responsibilities Job Descriptions for Big data or Cloud Engineer We are looking for candidates with hands on experience in pyspark with GCP cloud. Qualifications 3-10 years of IT experience range is preferred. Able to effectively use GCP managed services e.g. Dataproc, Dataflow, pub/sub, Cloud functions, Big Query, GCS - At least 4 of these Services. Good to have knowledge on Cloud Composer, Cloud SQL, Big Table, Cloud Function. Strong experience in Big Data technologies – Hadoop, Sqoop, Hive and Spark including DevOPs. Good hands on expertise on either Python or Java programmi...

Job Title: Lead Data Engineer Job Summary The Lead Data Engineer will provide technical expertise in analysis, design, development, rollout and maintenance of data integration initiatives. This role will contribute to implementation methodologies and best practices, as well as work on project teams to analyse, design, develop and deploy business intelligence / data integration solutions to support a variety of customer needs. This position oversees a team of Data Integration Consultants at various levels, ensuring their success on projects, goals, trainings and initiatives though mentoring and coaching. Provides technical expertise in needs identification, data modelling, data movement and t...

Essential for this role: Education and Qualifications: • Bachelor’s degree in IT, Computer Science, Software Engineering, Business Analytics or equivalent. • Minimum seven plus years of experience in data analytics field • Experience with Azure/AWS Databricks • Experience in building and optimizing data pipelines, architectures and data sets • Excellent experience in Scala or Python, PySpark and SQL • Ability to troubleshoot and optimize complex queries on the Spark platform • Knowledgeable on structured and unstructured data design / modelling, data access and data storage techniques • Expertise in designing and deploying data applications on cloud solutions, such as Azure or AWS • Hands on...

As a Data Engineering Senior Software Engineer / Tech Lead / Senior Tech Lead at a leading digital health platform in Navi Mumbai, Ghansoli, you will play a crucial role in designing, building, and optimizing ETL/ELT pipelines using tools like Pentaho, Talend, or similar. Your responsibilities will include working on traditional databases such as PostgreSQL, MSSQL, Oracle, as well as MPP/modern systems like Vertica, Redshift, BigQuery, and MongoDB. You will collaborate cross-functionally with BI, Finance, Sales, and Marketing teams to define data needs and participate in data modeling, data quality checks, and data integration. Additionally, you will be implementing solutions involving messa...

Hands on knowledge of SPARK, scala, etc. Hands on knowledge of RDBMS like MS-SQL/Oracle Knowledge of CI/CD tools like uDeploy Understanding of Vanguard and its landscape will be additional advantage

As a Senior Cloud Data Platform (GCP) Engineer at Incedo, you will be responsible for managing and optimizing the Google Cloud Platform environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or Azure and have experience with big data technologies such as Hadoop or Spark. You will be responsible for configuring and optimizing the GCP environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security team to ensure that th...

Title : BigData Developer Location : Bangalore Position type : Full time Requirement: No Remote: All 5 days work from office. Working experience of Hadoop, Hive SQLs, Spark, Bigdata Eco System Tools. • Should be able to tweak queries and work on performance enhancement. • The candidate will be responsible for delivering code, setting up environment, connectivity, deploying the code in production after testing. • The candidate should have strong functional and technical knowledge to deliver what is required and he/she should be well acquainted with Banking terminologies. Occasionally, the candidate may have to be responsible as a primary contact and/or driver for small to medium size projects...

Role Description As a Senior Cloud Data Platform (GCP) Engineer at Incedo, you will be responsible for managing and optimizing the Google Cloud Platform environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or Azure and have experience with big data technologies such as Hadoop or Spark. You will be responsible for configuring and optimizing the GCP environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security team ...

Role Description As a Senior Cloud Data Platform (GCP) Engineer at Incedo, you will be responsible for managing and optimizing the Google Cloud Platform environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or Azure and have experience with big data technologies such as Hadoop or Spark. You will be responsible for configuring and optimizing the GCP environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security team ...

Hive/Python/Spark Technical hands on data processing Database SQL knowledge for retrieval of data - transformation queries such as joins (full, left , right) , ranking , group by Database SQL/Oracle Good Communication skills. Additional skills - GitHub , Jenkins , shell scripting would be added advantage, large-scale data processing model implementation, and performance tracking Mandatory Competencies Big Data - Big Data - HIVE Big Data - Big Data - Pyspark Big Data - Big Data - SPARK Data Science and Machine Learning - Data Science and Machine Learning - Data Analyst Database - Oracle - PL/SQL Packages Programming Language - Python - Python Shell Beh - Communication and collaboration

Clear concept of dimensional data modelling(Logical), SCDs, Normalisation and denormalizations. Thoroughly translating the logical models to physical and data flows per the business requirements. Update and optimise the local and metadata models. Could you evaluate the implemented data system for the variances, discrepancies and efficiencies? Troubleshoot and optimise the existing data flows, models and processing jobs by modularising. Explore ways to enhance the data quality and reliability. Strong in writing UDF Previous knowledge of implementing the DQ framework on Spark would be an added advantage. Good in writing programming using Python/Scala/PySpark is a must. Strong knowledge of spar...

6 + years of overall IT experience in Telecom OSS especially in Assurance domain Solution, design, and Implementation - Strong Knowledge of Telecom OSS domain, with excellent experience Service Now for Assuance - Knowledge and experience on Big Data,Data lake solution, KAFKA , Hadoop/Hive. - Experience on Python (pyspark) is essential. - Implementation experience in continuous integration and delivery philosophies and practices specifically on Docker, Git, JenKins - Self driven and highly motivated candidate for a client facing role in a challenging environment

Hands On with advanced SQL, Python etc. Hands On in data profiling Hands On in working on Cloud like Azure and Cloud DW like Databricks Hands On experience on Scheduling tools like Airflow, Control M etc. Knowledgeable on Big Data tools like Spark (python/scala), Hive, Impala, Hue and storage (e.g. HDFS, HBase) Knowledgeable in CICD processes Bitbucket/GitHub, Jenkins, Nexus etc. Knowledgeable managing structured and unstructured data types

Role:Sr Python Developer Must have:Python Spark/Postgres SQL ,Spark / Apache Spark (AWS) Strong development experience using Python and SQL on AWS using Glue, Lambda.

Description: Understanding of data modeling concepts data warehousing tools and databases Experience in Snowflake SQL Cloud (AWS) Unix Technologies experience Must have hands on experience on SQL Snowflake Experience in AWS S3 and other common aws services. Should have experience working with multiple file formats especially with Parquet and Avro Should have Knowledge on Jenkins Git Udeploy CICD tools. Should have Experience on SQL writting and Unix commands Good communication skill and good team player Good to have Agile knowledge Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade :B Level :To Be Defined Named Job Posting? (if Yes - needs to be approv...

Description Skills: Proficiency in SQL is a must. PL/SQL to understand integration SP part. Experience in PostgreSQL is must. Basic knowledge of Google Cloud Composer ( or Apache Airflow). Composer is managed GCP service for Apache Airflow. All pipelines are orchestrated and scheduled through Composer GCP basics-high level understanding of using GCP UI and services like Cloud SQL PostgreSQL Cloud Composer Cloud Storage Dataproc Airlfow DAGs are written in Python basic knowledge of Python code for DAGs Dataproc is Managed Spark in GCP so a bit of PySpark knowledge is also nice to have. Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade C Level To Be Def...

Description 1.Hands on industry experience in design and coding from scratch in AWS Glue-Pyspark with services like S3 DynamoDB StepFunctions etc. 2.Hands on industry experience in design and coding from scratch in Snowflake 3.Experience in Pyspark/Snowflake 1 to 3 years with overall around 5 years of experience in building data/analytics solutions Level Senior Consultant or below Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade C Level To Be Defined Named Job Posting? (if Yes - needs to be approved by SCSC) No Remote work possibility Yes Global Role Family 60236 (P) Software Engineering Local Role Name 6361 Software Engineer Local Skills 59383 AWS G...

JD -7+ years of hands on experience in Python especially dealing with Pandas and Numpy Good hands-on experience in Spark PySpark and Spark SQL Hands on experience in Databricks Unity Catalog Delta Lake Lake house Platform Medallion Architecture Azure Data Factory ADLS Experience in dealing with Parquet and JSON file format Knowledge in Snowflake.