2480 Sqoop Jobs - Page 4

Design, develop, and maintain a robust, end-to-end data analytics pipeline for real-time data processing and visualization Take ownership of the entire project lifecycle, from architectural design and component integration to deployment, security, and presentation Work on data analytics platform (on GKE/GDC air gapped) to provide real-time hot spot prediction and ride recommendations for taxi drivers The application will use the NYC taxi dataset and showcase the capabilities of Kafka, Spark, HBase, Solr, Trino, Apache Airflow, Apache Superset and Apache Ranger Requirements Must Have: Big Data Ecosystem: Apache Kafka, Apache Spark, HBase, Apache Solr, and TrinoTools & Orchestration: Apache Ai...

Job Summary: We are looking for an experienced GCP Data Engineer to design, build, and optimize data pipelines and solutions on the Google Cloud Platform (GCP) . The ideal candidate will have deep expertise in BigQuery, Dataflow, Dataproc , and other GCP data services, along with strong programming and analytical skills. Key Responsibilities: Design and develop scalable data pipelines and ETL workflows using GCP services such as BigQuery, Dataflow, Dataproc, Composer, and Pub/Sub . Build and maintain data lakes, data warehouses, and analytics solutions on GCP. Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and structured datasets . Implement data ing...

Job Summary: We are looking for an experienced GCP Data Engineer to design, build, and optimize data pipelines and solutions on the Google Cloud Platform (GCP) . The ideal candidate will have deep expertise in BigQuery, Dataflow, Dataproc , and other GCP data services, along with strong programming and analytical skills. Key Responsibilities: Design and develop scalable data pipelines and ETL workflows using GCP services such as BigQuery, Dataflow, Dataproc, Composer, and Pub/Sub . Build and maintain data lakes, data warehouses, and analytics solutions on GCP. Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and structured datasets . Implement data ing...

Overview Prodapt is looking for senior data engineers having 3+ years of experience having hands on expertise on GCP, BQ, Data Pipeline creation & ETL. Responsibilities Understanding the requirements Performing in-depth analysis on data Setting up secure data pipelines from ingestion to delivery Individual contributor Requirements 3- 5 years of experience in Data Engineering with strong ability to work independently, communicate effectively, and deliver high-quality solutions. Should have strong SQL programming skills. Experience in cloud-based systems (GCP, BQ) is must. Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues. Expertise...

Job Summary: We are looking for a Big Data Engineer with strong expertise in Apache Spark and Scala to design, develop, and optimize large-scale data processing solutions. The ideal candidate will have hands-on experience with distributed data systems, ETL development, and modern data engineering tools. Key Responsibilities: Design and develop data ingestion, transformation, and processing pipelines using Apache Spark (Core, SQL, Streaming) and Scala . Build scalable and reliable big data solutions on distributed platforms such as Hadoop, Databricks, or EMR . Optimize Spark jobs for performance and scalability . Work closely with data architects, analysts, and business teams to define data r...

Job Summary: We are seeking an experienced ML Data Scientist to design, develop, and deploy machine learning models that solve complex business problems and deliver measurable outcomes. The ideal candidate will possess strong analytical skills, deep understanding of statistical and ML techniques, and hands-on experience in model development and deployment in production environments. Key Responsibilities: Understand business objectives and translate them into data-driven solutions . Design, develop, and implement machine learning models, algorithms, and predictive analytics . Perform data cleaning, preprocessing, feature engineering , and exploratory data analysis (EDA). Evaluate model perfor...

Job Summary: We are looking for an experienced GCP Data Engineer to design, build, and optimize data pipelines and solutions on the Google Cloud Platform (GCP) . The ideal candidate will have deep expertise in BigQuery, Dataflow, Dataproc , and other GCP data services, along with strong programming and analytical skills. Key Responsibilities: Design and develop scalable data pipelines and ETL workflows using GCP services such as BigQuery, Dataflow, Dataproc, Composer, and Pub/Sub . Build and maintain data lakes, data warehouses, and analytics solutions on GCP. Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and structured datasets . Implement data ing...

Job Title: Hadoop Administrator Location: Chennai, India Experience: 4+yrs of experience in IT, with At least 3+ years of experience with cloud and system administration. At least 3 years of experience with and strong understanding of big data technologies in Hadoop ecosystem Hive, HDFS, Map/Reduce, Flume, Pig, Cloudera, HBase Sqoop, Spark etc. Company: Smartavya Analytica Private limited is a niche Data and AI company. Based in Pune, we are pioneers in data-driven innovation, transforming enterprise data into strategic insights. Established in 2017, our team has experience in handling large datasets up to 20 PBs in a single implementation, delivering many successful data and AI projects acr...

Spark, Hive, Scala, SQL ,Python scripting ,data visualization Key Responsibilities: Leadership and Team Management: Lead and mentor a team of data engineers, fostering a culture of collaboration and continuous improvement. Oversee the planning and execution of data projects, ensuring alignment with business objectives and timelines. Provide technical guidance and expertise to the team, promoting best practices in data engineering. Data Architecture and Development: Design, develop, and maintain scalable data pipelines and ETL processes to ingest, process, and store large volumes of data. Architect and implement robust data models and schemas to support analytics and business intelligence nee...

What will your role look like Design, develop, and maintain scalable data pipelines and applications using Python and PySpark. Build end-to-end data solutions from scratch, ensuring clean, modular, and reusable code. Write and optimize complex SQL queries for large-scale data processing, transformation, and validation. Develop dynamic, reusable frameworks and implement design patterns for scalability and performance. Refactor legacy codebases to enhance performance, maintainability, and reliability. Collaborate with cross-functional teams including data engineers, analysts, and stakeholders to translate business requirements into technical solutions. Utilize Databricks and Azure Data Factory...

Description Role : Hadoop Admin Manager CDP. Years of Experience : 10-15 Yrs. Location : Mumbai (Kurla). Shifts : 24-7 (Rotational Shift). Mode : Onsite. Experience : 10+ yrs of experience in IT, with At least 7+ years of experience with cloud and system administration. At least 5 years of experience with and strong understanding of big data technologies in Hadoop ecosystem Hive, HDFS, Map/Reduce, Flume, Pig, Cloudera, HBase Sqoop, Spark etc. Company : Smartavya Analytica Private limited is a niche Data and AI company. Based in Pune, we are pioneers in data-driven innovation, transforming enterprise data into strategic insights. Established in 2017, our team has experience in handling large ...

AIA-Pune Job Summary Data Engineer 4 to 9 Years Responsibilities Data Engineering design experience (to support with HLD review LLD build out etc.) Azure experience confident in working with Azure Cloud and Data Engineering capabilities ( e.g. ADLS Gen2 Synapse ADF etc.) Advanced SQL experience query writing & optimisation data aggregation stored procedures etc. Experience writing data Pipelines using Python and PySpark. Relational data model / data schema design experience Understanding of DevOps and CI/CD in the context of Data Engineering Familiarity with working in a scrum environment and supporting scrum ceremonies Certifications Required Azure Pyspark Python

Overview Should have a strong understanding of Kafka and the Elk (Elasticsearch, Logstash, Kibana) stack, including hands-on experience in implementing and maintaining these technologies Strong background in scripting and programming languages like Python, Java is essential for building and maintaining Kafka and Elk based solutions. In-depth knowledge and hands-on experience utilizing Log Shipper tools such as Fluentd, Fluent Bit, and Filebeat. Candidate should have excellent troubleshooting and problem-solving skills to identify and resolve issues in the Data Pipeline and ensure smooth operation. Investigate and resolve complex issues related to log data, log parsing, and ELK Stack performa...

Overview Prodapt is lookig for Senior Technical Lead in Data Engineering stream having 11+ years of experience having hands on expertise on GCP, BQ, Data Pipeline creation & ETL. Responsibilities Understanding the requirements Performing in-depth analysis on data Setting up secure data pipelines from ingestion to delivery Individual contributor Requirements 11+ years of experience in Data Engineering with strong ability to work independently, communicate effectively, and deliver high-quality solutions. Should have strong SQL programming skills. Experience in cloud-based systems (GCP, BQ) is must. Performs data analysis required to troubleshoot data related issues and assist in the resolution...

About The Role Job Title Data Science/Data Engineering ML9 - Sales Excellence COE - Data Engineering Specialist Management Level :ML9 Location:Open Must have skills: strong GCP cloud technology experience, Big query, Data Science basics. Good to have skills: Build and maintain data models from different sources. Experience: Minimum 5 year(s) of experience is required Educational Qualification: Graduate/Post graduate Job Summary :The Center of Excellence (COE) makes sure that the sales and pricing methods and offerings of Sales Excellence are effective. The COE supports salespeople through its business partners and Analytics and Sales Operations teams. The Data Engineer helps manage data sour...

Skills: Azure, Data Engineer, python, SQL, ADF, Databrick, Azure Databricks,. Position:Azure Data Engineer. Work Experience:5-8years. Work location:Bangalore/Hyderabad/Pune. Required Skills. Azure, Strong SQL, ADF, Python, PySpark, Databricks. Key Responsibilities. Design and develop data pipelines using Azure services. Collaborate with cross-functional teams to understand data requirements. Optimize and troubleshoot data pipelines for performance and reliability. Implement data security and governance best practices. Stay updated with the latest industry trends and emerging Azure. Show more Show less

We are looking for a skilled Business Technology Solutions Consultant with 5-10 years of experience to join our team in India. The ideal candidate will have a strong background in Big Data Development and be able to work on innovative solutions. Roles and Responsibility Design, develop, and implement Big Data solutions using various technologies. Collaborate with cross-functional teams to identify business requirements and develop technical solutions. Develop and maintain large-scale data processing systems and pipelines. Ensure data quality, integrity, and security in all developed systems. Troubleshoot and resolve complex technical issues related to Big Data development. Stay updated with ...

The candidate will work on modernizing high-volume, large-scale, multi-tiered transaction processing production system to cloud. Working in Agile software development lifecycle methodology, he will analyze the requirements, provide solution and mentor junior developers. Will design and develop near real-time applications using various Java based cloud technologies. She / He will work as technical lead, providing solution for modernization effort. He / She will also analyze new requirements and provide estimates for new development, mentor team member and in charge for the technical delivery of the project / module. Very good verbal and written communication 10+ Years of experience in develop...

As a Senior Cloud Data Platform (Azure) Engineer at Incedo, you will be responsible for managing and optimizing the Microsoft Azure environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or GCP and have experience with data warehousing technologies such as Azure Synapse Analytics. You will be responsible for configuring and optimizing the Azure environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security team to en...

As part of the Data Engineering team, you will serve as a Tier 2 Support Specialist responsible for managing and resolving technical support tickets related to our data pipelines, ingestion processes, and data quality. You will play a critical role in ensuring the stability and reliability of our intelligent data platform, supporting business users and internal stakeholders with timely solutions. Your work will directly contribute to smooth operations, improved client service, and enabling data-driven decision-making across Cushman & Wakefield. About the Role: Serve as the Tier 2 escalation point for support tickets related to data ingestion, pipelines, and data marts. Troubleshoot and resol...

Job Description Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Do Oversee and support process by reviewing daily transactions on performance parameters Review performance dashboard and the scores for the team Support the team in improving performance parameters by providing technical support and process guidance Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions Ensure standard processes and procedures are followed to resolve all client qu...

Job Description Project Manager with Splunk suite Implementation experience A Project Manager (PM) for Splunk suite implementation needs to oversee projects from initiation to completion, ensuring the Splunk solution meets business requirements by managing timelines, budgets, resources, and stakeholders. Key responsibilities include guiding the implementation team, facilitating communication between stakeholders and the technical team, managing risk, and driving the adoption of the Splunk solution through user training. A strong understanding of Splunk's architecture, dashboard development, data ingestion, and the use of Splunk Query Language (SPL) is also essential for effective project ove...

Job Description Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Do Oversee and support process by reviewing daily transactions on performance parameters Review performance dashboard and the scores for the team Support the team in improving performance parameters by providing technical support and process guidance Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions Ensure standard processes and procedures are followed to resolve all client qu...

Job Description Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Do Oversee and support process by reviewing daily transactions on performance parameters Review performance dashboard and the scores for the team Support the team in improving performance parameters by providing technical support and process guidance Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions Ensure standard processes and procedures are followed to resolve all client qu...

Adobe Campaign Role Purpose Provide advisory, consulting, and technical implementation services to customers on Adobe Experience Platform and Adobe Journey Optimizer Assess customer requirements, pain points, goals and make recommendations on solution architecture, design and roadmap Configure Adobe Experience Platform services like Identity Service, Data Storage Layer, Profile dataset, Real-time Customer Profile etc. Implement data ingestion from various sources like CRM, MPNs, Website data, Mobile apps using APIs, Experience Data Model (XDM), schemas and mappers Create customer, visitor and product audiences using segments and AI/ML powered segments Configure destinations to activate audie...