756 Apache Pig Jobs - Page 29

The ideal candidate will have expertise in integrating Kafka with various systems and technologies. Roles and Responsibility Design, develop, and implement scalable and efficient Kafka integration solutions. Collaborate with cross-functional teams to identify and prioritize integration requirements. Develop and maintain technical documentation for Kafka integrations. Troubleshoot and resolve issues related to Kafka integration. Ensure compliance with industry standards and best practices for data security and privacy. Participate in code reviews and contribute to the improvement of the overall code quality. Job Requirements Strong understanding of Kafka architecture and its ecosystem. Experi...

Technologies you will get to work with: - 1.Azure Data-bricks 2.Azure Data Factory 3.Azure DevOps 4.Spark with Python & Scala and Airflow scheduling. What You will Do: - Build large-scale batch and real-time data pipelines with data processing frameworks like spark, Scala on Azure platform. Collaborate with other software engineers, ML engineers and stakeholders, taking learning and leadership opportunities that will arise every single day. Use best practices in continuous integration and delivery. Sharing technical knowledge with other members of the Data Engineering Team. Work in multi-functional agile teams to continuously experiment, iterate and deliver on new product objectives. You wil...

Technologies you will get to work with: - 1.Azure Data-bricks 2.Azure Data Factory 3.Azure DevOps 4.Spark with Python & Scala and Airflow scheduling. What You will Do: - Build large-scale batch and real-time data pipelines with data processing frameworks like spark, Scala on Azure platform. Collaborate with other software engineers, ML engineers and stakeholders, taking learning and leadership opportunities that will arise every single day. Use best practices in continuous integration and delivery. Sharing technical knowledge with other members of the Data Engineering Team. Work in multi-functional agile teams to continuously experiment, iterate and deliver on new product objectives. You wil...

Roles and Responsibilities: 4+ years of experience as a data developer using Python Knowledge in Spark, PySpark preferable but not mandatory Azure Cloud experience (preferred) Alternate Cloud experience is fine preferred experience in Azure platform including Azure data Lake, data Bricks, data Factory Working Knowledge on different file formats such as JSON, Parquet, CSV, etc. Familiarity with data encryption, data masking Database experience in SQL Server is preferable preferred experience in NoSQL databases like MongoDB Team player, reliable, self-motivated, and self-disciplined.

12+ Years experience in Big data Space across Architecture, Design, Development, testing & Deployment, full understanding in SDLC. 1. Experience of Hadoop and related technology stack experience 2. Experience of the Hadoop Eco-system(HDP+CDP) / Big Data (especially HIVE) Hand on experience with programming languages such as Java/Scala/python Hand-on experience/knowledge on Spark 3. Being responsible and focusing on uptime and reliable running of all or ingestion/ETL jobs 4. Good SQL and used to work in a Unix/Linux environment is a must. 5. Create and maintain optimal data pipeline architecture. 6. Experience performing root cause analysis on internal and external data and processes to answe...

Roles & Responsibilities: Total 8-10 years of working experience Experience/Needs: 8-10 Years of experience with big data tools like Spark, Kafka, Hadoop etc. Design and deliver consumer-centric high performant systems. You would be dealing with huge volumes of data sets arriving through batch and streaming platforms. You will be responsible to build and deliver data pipelines that process, transform, integrate and enrich data to meet various demands from business Mentor team on infrastructural, networking, data migration, monitoring and troubleshooting aspects Focus on automation using Infrastructure as a Code (IaaC), Jenkins, devOps etc. Design, build, test and deploy streaming pipelines f...

Cloud Data Engineer Req number: R5934 Employment type: Full time Worksite flexibility: Remote Who we are CAI is a global technology services firm with over 8,500 associates worldwide and a yearly revenue of $1 billion+. We have over 40 years of excellence in uniting talent and technology to power the possible for our clients, colleagues, and communities. As a privately held company, we have the freedom and focus to do what is right—whatever it takes. Our tailor-made solutions create lasting results across the public and commercial sectors, and we are trailblazers in bringing neurodiversity to the enterprise. Job Summary We are seeking a motivated Cloud Data Engineer that has experience in bu...

Strong problem-solving skills with a focus on product development. Domain expertise in Big Data, Data Platforms, and Distributed Systems. Proficiency in Java, Scala, or Python (hands-on experience with Apache Spark is essential). Experience with data ingestion frameworks such as Apache Storm, Flink, or Spark Streaming. Experience with streaming technologies like Kafka, Kinesis, Oplogs, Binlogs, or Debezium. Strong database skills with experience in HDFS, Delta Lake, Iceberg, or Lakehouse architectures.

Mandatory Skills: Microsoft Azure, Py spark, Data Bricks, Hadoop, Spark, Airflow, Kafka Requirments: Experience working with distributed technology tools for developing Batch and Streaming pipelines using. SQL, Spark, Python Airflow Scala Kafka Experience in Cloud Computing, e.g., AWS, GCP, Azure, etc. Able to quickly pick up new programming languages, technologies, and frameworks. Strong skills building positive relationships across Product and Engineering. Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders Experience with creating/ configuring Jenkins pipeline for smooth CI/CD process for Managed Spark jobs, build Docker im...

Overview Data Science Team works in developing Machine Learning (ML) and Artificial Intelligence (AI) projects. Specific scope of this role is to develop ML solution in support of ML/AI projects using big analytics toolsets in a CI/CD environment. Analytics toolsets may include DS tools/Spark/Databricks, and other technologies offered by Microsoft Azure or open-source toolsets. This role will also help automate the end-to-end cycle with Azure Pipelines. You will be part of a collaborative interdisciplinary team around data, where you will be responsible of our continuous delivery of statistical/ML models. You will work closely with process owners, product owners and final business users. Thi...

Immediate job opening for # Python+SQL_C2H_Pan India. #Skill:Python+SQL #Job description: Strong programming skills in Python programming and advance SQL. strong experience in NumPy, Pandas, Data frames Strong analytical and problem-solving skills. Excellent communication and collaboration abilities.

Developer leads the cloud application development/deployment. A developer responsibility is to lead the execution of a project by working with a senior level resource on assigned development/deployment activities and design, build, and maintain cloud environments focusing on uptime, access, control, and network security using automation and configuration management tools Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Strong proficiency in Java, Spring Framework, Spring boot, RESTful APIs, excellent understanding of OOP, Design Patterns. Strong knowledge of ORM tools like Hibernate or JPA, Java based Micro-services framew...

Developer leads the cloud application development/deployment. A developer responsibility is to lead the execution of a project by working with a senior level resource on assigned development/deployment activities and design, build, and maintain cloud environments focusing on uptime, access, control, and network security using automation and configuration management tools Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Strong proficiency in Java, Spring Framework, Spring boot, RESTful APIs, excellent understanding of OOP, Design Patterns. Strong knowledge of ORM tools like Hibernate or JPA, Java based Micro-services framew...

Mandatory Skills: AWS, Python, SQL, spark, Airflow, Snowflake Responsibilities: Create and manage cloud resources in AWS Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies Data processing/transformation using various technologies such as Spark and Cloud Services. You will need to understand your part of business logic and implement it using the language supported by the base data platform Develop automated data quality check to make sure right data enters the platf...

Strong knowledge of UNIX systems and shell scripting. Proficiency in SQL for database management and manipulation. Hands-on experience with SPARK for big data processing. Methodologies: Understanding and application of Agile practices. Communication: Excellent verbal and written communication skills, with the ability to collaborate effectively with team members and stakeholders."Candidates should possess an experience between 3 to 5 years in data engineering. Strong knowledge in UNIX, SQL, SPARK. Good to have skills are - PySpark and AWS. Good communication and should know the Agile practices.

Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Do Oversee and support process by reviewing daily transactions on performance parameters Review performance dashboard and the scores for the team Support the team in improving performance parameters by providing technical support and process guidance Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions Ensure standard processes and procedures are followed to resolve all client queries Resolve cl...

Required Skills: 5-8 years' experience candidate as back end - data engineer. Strong experience in SQL. Strong knowledge and experience Python and Py Spark. Experience in AWS. Experience in Docker and OpenShift. Hands on experience with REST Concepts. Design and Develop business solutions on the data front. Experience in implementation of new enhancements and also handling defect triage. Candidate must have strong analytical abilities. Skills/ Competency Additionally Preferred Jira, Bit Bucket Experience on Kafka. Experience on Snowflake. Domain knowledge in Banking. Analytical skills. Excellent communication skills Working knowledge of Agile. Your future duties and responsibilities: Design ...

Why this job matters This role holder will be a lead Software Engineer manager responsible for by co-ordinating between OSS designers, network design, and wider OSS/BSS stakeholders and lead the software application(s). The role holder is expected to possess good knowledge of Network Performance Management functions, Service Assurance & OSS domain. Successful candidates need to have hands on technical experience in Data Science, Java & Big Data based technology, NoSQL Database, Streaming technologies such as Kafka and Linux platform. Should have good understanding of Network performance management tool sets. Candidate should have prior experience in telecom service provider or with system in...

Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Do Oversee and support process by reviewing daily transactions on performance parameters Review performance dashboard and the scores for the team Support the team in improving performance parameters by providing technical support and process guidance Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions Ensure standard processes and procedures are followed to resolve all client queries Resolve cl...

Skills : Bigdata, Pyspark, Hive, Spark Optimization Good to have : GCP Roles and Responsibilities Skills : Bigdata, Pyspark, Hive, Spark Optimization Good to have : GCP

We are looking for a highly skilled Data Engineer with 6 to 9 years of experience to join our team at BlackBaud, located in [location to be specified]. The ideal candidate will have a strong background in data engineering and excellent problem-solving skills. Roles and Responsibility Design, develop, and implement data pipelines and architectures to support business intelligence and analytics. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems, ensuring scalability, reliability, and performance. Troubleshoot and resolve complex technical issues related to data engineering projects. Participate in code reviews...

Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : AWS BigData Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project milestones are met, facilitating discussions to address challenges, and guiding your team in implementing effective solutions. You will also engage in strategi...

As an Associate Software Developer at IBM, you'll work with clients to co-createsolutions to major real-world challenges by using best practice technologies, tools,techniques, and products to translate system requirements into the design anddevelopment of customized systems Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Strong proficiency in Java, Spring Framework, Spring boot, RESTful APIs, excellent understanding of OOP, Design Patterns. Strong knowledge of ORM tools like Hibernate or JPA, Java based Micro-services framework, Hands on experience on Spring boot Microservices, Primary Skills: - Core Java, Spring Boot, Ja...

As an Associate Software Developer at IBM, you'll work with clients to co-create solutions to major real-world challenges by using best practice technologies, tools, techniques, and products to translate system requirements into the design and development of customized systems Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Strong proficiency in Java, Spring Framework, Spring boot, RESTful APIs, excellent understanding of OOP, Design Patterns. Strong knowledge of ORM tools like Hibernate or JPA, Java based Micro-services framework, Hands on experience on Spring boot Microservices, Primary Skills: - Core Java, Spring Boot,...

Developer leads the cloud application development/deployment. A developer responsibility is to lead the execution of a project by working with a senior level resource on assigned development/deployment activities and design, build, and maintain cloud environments focusing on uptime, access, control, and network security using automation and configuration management tools Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Strong proficiency in Java, Spring Framework, Spring boot, RESTful APIs, excellent understanding of OOP, Design Patterns. Strong knowledge of ORM tools like Hibernate or JPA, Java based Micro-services framew...