756 Apache Pig Jobs - Page 5

Your role and responsibilities As an Application Developer, you will lead IBM into the future by translating system requirements into the design and development of customized systems in an agile environment. The success of IBM is in your hands as you transform vital business needs into code and drive innovation. Your work will power IBM and its clients globally, collaborating and integrating code into enterprise systems. You will have access to the latest education, tools and technology, and a limitless career path with the worlds technology leader. Come to IBM and make a global impact Responsibilities: Responsible to manage end to end feature development and resolve challenges faced in impl...

We are looking for skilled Hadoop Developers with 5-10 years of experience to join our team in Bangalore, Kolkata, Hyderabad, and Pune. The ideal candidate should have strong proficiency in Hadoop, Scala, Spark, and SQL. Roles and Responsibility Design, develop, and implement scalable data processing systems using Hadoop. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data pipelines using Spark and Scala. Troubleshoot and resolve complex technical issues related to Hadoop. Participate in code reviews and ensure high-quality code standards. Stay updated with the latest trends and technologies in Hadoop development. Job...

We are looking for a skilled Hadoop professional with 7-14 years of experience to join our team in Pune and Hyderabad. The ideal candidate will have expertise in Hadoop, Spark, Kafka, Data Visualization, scripting languages in Python and R, SQL and NoSQL, Agile methodology, ETL / ELT data pipelines. Roles and Responsibility Design and develop scalable Hadoop applications using Spark and Kafka. Create data visualizations using tools like Splunk, Qlik, and Grafana. Develop and maintain ETL/ELT data pipelines using Agile methodology. Collaborate with cross-functional teams to identify and prioritize project requirements. Troubleshoot and resolve technical issues related to Hadoop applications. ...

We are looking for a skilled Hadoop Admin with 7 to 12 years of experience, located in Hyderabad, Bangalore, Chennai, and Bhubaneshwar. The ideal candidate will have hands-on experience in building EKS clusters, Spark on EKS, AWS (EMR, S3, RDS), Python as programming skills, Shell scripting, and any CI/CD. Roles and Responsibility Manage and maintain large-scale Hadoop clusters for high availability and performance. Design and implement scalable data processing pipelines using Spark and Airflow instances. Integrate Ranger and Hive Metastore services in EKS for secure data storage and access. Collaborate with cross-functional teams to identify and resolve technical issues. Develop and impleme...

As a Senior Cloud Data Platform (GCP) Engineer at Incedo, you will be responsible for managing and optimizing the Google Cloud Platform environment, ensuring its performance, scalability, and security You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments You will be skilled in cloud computing platforms such as AWS or Azure and have experience with big data technologies such as Hadoop or Spark You will be responsible for configuring and optimizing the GCP environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise You will also work with the security team to ensure that the GC...

Responsibilities Job Responsibilities: Collaborate closely with global product and engineering teams to design AI and LLM solutions and support business objectives. Design, develop and deliver AI/ML models, leveraging cutting edge technologies to drive innovation. Analyze user behavior on existing solutions and enhance them through an AI/ML driven approach. Basic Qualifications Bachelor or above degree in Computer Science or Engineering 6+ years of working experience in Natural Language Processing and machine learning. Deep knowledge in NLP techniques, including embeddings, attention mechanisms, sequence-to-sequence models, generative AI, etc. Rich experience in designing and implementing NL...

Design, develop, and maintain a robust, end-to-end data analytics pipeline for real-time data processing and visualization Take ownership of the entire project lifecycle, from architectural design and component integration to deployment, security, and presentation Work on data analytics platform (on GKE/GDC air gapped) to provide real-time hot spot prediction and ride recommendations for taxi drivers The application will use the NYC taxi dataset and showcase the capabilities of Kafka, Spark, HBase, Solr, Trino, Apache Airflow, Apache Superset and Apache Ranger Requirements Must Have: Big Data Ecosystem: Apache Kafka, Apache Spark, HBase, Apache Solr, and TrinoTools & Orchestration: Apache Ai...

Job Summary: We are looking for an experienced GCP Data Engineer to design, build, and optimize data pipelines and solutions on the Google Cloud Platform (GCP) . The ideal candidate will have deep expertise in BigQuery, Dataflow, Dataproc , and other GCP data services, along with strong programming and analytical skills. Key Responsibilities: Design and develop scalable data pipelines and ETL workflows using GCP services such as BigQuery, Dataflow, Dataproc, Composer, and Pub/Sub . Build and maintain data lakes, data warehouses, and analytics solutions on GCP. Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and structured datasets . Implement data ing...

Job Summary: We are looking for an experienced GCP Data Engineer to design, build, and optimize data pipelines and solutions on the Google Cloud Platform (GCP) . The ideal candidate will have deep expertise in BigQuery, Dataflow, Dataproc , and other GCP data services, along with strong programming and analytical skills. Key Responsibilities: Design and develop scalable data pipelines and ETL workflows using GCP services such as BigQuery, Dataflow, Dataproc, Composer, and Pub/Sub . Build and maintain data lakes, data warehouses, and analytics solutions on GCP. Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and structured datasets . Implement data ing...

Job Summary: We are looking for a Big Data Engineer with strong expertise in Apache Spark and Scala to design, develop, and optimize large-scale data processing solutions. The ideal candidate will have hands-on experience with distributed data systems, ETL development, and modern data engineering tools. Key Responsibilities: Design and develop data ingestion, transformation, and processing pipelines using Apache Spark (Core, SQL, Streaming) and Scala . Build scalable and reliable big data solutions on distributed platforms such as Hadoop, Databricks, or EMR . Optimize Spark jobs for performance and scalability . Work closely with data architects, analysts, and business teams to define data r...

Job Summary: We are looking for an experienced GCP Data Engineer to design, build, and optimize data pipelines and solutions on the Google Cloud Platform (GCP) . The ideal candidate will have deep expertise in BigQuery, Dataflow, Dataproc , and other GCP data services, along with strong programming and analytical skills. Key Responsibilities: Design and develop scalable data pipelines and ETL workflows using GCP services such as BigQuery, Dataflow, Dataproc, Composer, and Pub/Sub . Build and maintain data lakes, data warehouses, and analytics solutions on GCP. Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and structured datasets . Implement data ing...

Job Title: Hadoop Administrator Location: Chennai, India Experience: 4+yrs of experience in IT, with At least 3+ years of experience with cloud and system administration. At least 3 years of experience with and strong understanding of big data technologies in Hadoop ecosystem Hive, HDFS, Map/Reduce, Flume, Pig, Cloudera, HBase Sqoop, Spark etc. Company: Smartavya Analytica Private limited is a niche Data and AI company. Based in Pune, we are pioneers in data-driven innovation, transforming enterprise data into strategic insights. Established in 2017, our team has experience in handling large datasets up to 20 PBs in a single implementation, delivering many successful data and AI projects acr...

Spark, Hive, Scala, SQL ,Python scripting ,data visualization Key Responsibilities: Leadership and Team Management: Lead and mentor a team of data engineers, fostering a culture of collaboration and continuous improvement. Oversee the planning and execution of data projects, ensuring alignment with business objectives and timelines. Provide technical guidance and expertise to the team, promoting best practices in data engineering. Data Architecture and Development: Design, develop, and maintain scalable data pipelines and ETL processes to ingest, process, and store large volumes of data. Architect and implement robust data models and schemas to support analytics and business intelligence nee...

Skills: Kafka, Apache Kafka, Li-apache-kafka-clients, Kafka-tools, Kafka Streams, Kafka-monitor,. Company Overview. CN Solutions is a leading IT services and IT consulting company that specializes in providing staffing solutions, leadership hiring, and other IT-related services. With a team of experienced professionals, CN Solutions has a strong expertise in web technologies, databases, software testing, CRM, ERP, data warehousing, middleware, ESM, business analysis, cyber security, IT infrastructure, and SAP hiring. Job Overview. We are hiring a Kafka professional with 4 to 6 years of experience to join our team at CN Solutions. As a Kafka expert, you will be responsible for designing, deve...

We are looking for a skilled Business Technology Solutions Consultant with 5-10 years of experience to join our team in India. The ideal candidate will have a strong background in Big Data Development and be able to work on innovative solutions. Roles and Responsibility Design, develop, and implement Big Data solutions using various technologies. Collaborate with cross-functional teams to identify business requirements and develop technical solutions. Develop and maintain large-scale data processing systems and pipelines. Ensure data quality, integrity, and security in all developed systems. Troubleshoot and resolve complex technical issues related to Big Data development. Stay updated with ...

The candidate will work on modernizing high-volume, large-scale, multi-tiered transaction processing production system to cloud. Working in Agile software development lifecycle methodology, he will analyze the requirements, provide solution and mentor junior developers. Will design and develop near real-time applications using various Java based cloud technologies. She / He will work as technical lead, providing solution for modernization effort. He / She will also analyze new requirements and provide estimates for new development, mentor team member and in charge for the technical delivery of the project / module. Very good verbal and written communication 10+ Years of experience in develop...

We are seeking a highly skilled Sr Engineer Consultant in Mobile Engineering. The successful candidate will be responsible for designing, implementing, and managing mobile device management (MDM) solutions using Intune and Azure. This person will create and maintain secure and scalable solutions for managing mobile devices, applications, and data. The Engineer will stay current with industry trends and emerging technologies to develop solutions that support and grow our business. Primary Responsibilities: Configure and deploy policies, profiles, compliance settings, and app protection policies for mobile devices and applications Monitor and troubleshoot device and app issues, as well as repo...

ole Description : As a SCALA Tech Lead, you will be a technical leader and mentor, guiding your team to deliver robust and scalable solutions. You will be responsible for setting technical direction, ensuring code quality, and fostering a collaborative and productive team environment. Your expertise in SCALA and your ability to translate business requirements into technical solutions will be crucial for delivering successful projects. Responsibilities : - Understand and implement tactical or strategic solutions for given business problems. - Discuss business needs and technology requirements with stakeholders. - Define and derive strategic solutions and identify tactical solutions when neces...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Apache Spark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and impleme...

About The Role Project Role : Application Designer Project Role Description : Assist in defining requirements and designing applications to meet business process and application requirements. Must have skills : Alteryx Good to have skills : Hadoop Administration Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Designer, you will assist in defining requirements and designing applications to meet business process and application requirements. Your typical day will involve collaborating with various stakeholders to gather insights, analyzing user needs, and translating them into functional specifications. You will en...

About The Role Job Title - Utilities Data Strategy/Architect Consultant Management Level : 09 - Consultant Location: Mumbai/ Pune/ Bangalore/ Gurgaon/Hyderabad. Must have skills: +5 years experience in a consulting/industry role with utilities exposure delivering data driven capabilities and solutions 3+ years of implementation experience in Asset Investment Planning tools (IBM Maximo/APM, Copperleaf, SAP IAM, Bentley APM, Aveva APM, Infor APM, GE Predix) 2+ years experience in designing and utilizing analytic solutions in enterprise-wide organizations or in a role defining data architecture for a cloud native solution or journey to cloud Experience guiding and managing a team of practitione...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Apache Spark Good to have skills : NA Minimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and imple...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Apache Spark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and impleme...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Apache Spark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and impleme...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Apache Spark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and impleme...