654 Apache Pig Jobs - Page 4

Educational Requirements Bachelor of Engineering Service Line Data & Analytics Unit Responsibilities Team Management: Lead and manage a team of technical professionals Data Engineer and Application Developers, ensuring effective collaboration and productivity. Client Interaction: Serve as the primary point of contact for clients, understanding their needs and ensuring successful project delivery. Status Reporting: Prepare and present detailed status reports on ongoing projects to stakeholders, highlighting progress, risks, and mitigation plans. Issue Resolution: Proactively identify and resolve any technical challenges or roadblocks faced by the team, ensuring smooth project execution. Proje...

At EY, youll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And were counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. EY GDS Data and Analytics (D&A) Manager - Palantir Job Overview Big Data Developer/Senior Data Engineer with 8 to 11 years of experience who would display strong analytical, problem-solving, programming, Business KPIs understanding and communication skills. They should be self-learner, detail-oriented team members who can consistently meet dead...

At EY, youll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And were counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. Position Name Data Engineer Position Level Senior Position Details EYs GDS Assurance Digital teams mission is to develop, implement and integrate technology solutions that better serve our audit clients and engagement teams. As a member of EYs core Assurance practice, youll develop a deep Audit related technical knowledge and outstanding databa...

Description: Overall 10+ years of experience in Python and Shell Knowledge of distributed systems like Hadoop and Spark as well as cloud computing platforms such as Azure and AWS Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade :B Level :To Be Defined Named Job Posting? (if Yes - needs to be approved by SCSC) :No Remote work possibility :No Global Role Family :To be defined Local Role Name :To be defined Local Skills :Ruby;automation;Python Languages Required::ENGLISH Role Rarity :To Be Defined

Description: About The Role :: Minimum 5 years of experience in a data-driven information environment, designing and implementing analytic/data systems, ideally in a IOT context Detail-oriented and comfortable working with requirements that are open-ended Experience and familiarity with big data tools and services, such as Kafka, AWS, S3, Flink, Spark, Lambda, Experience working in a cloud-based environment, ideally familiar with the AWS toolset Expertise in one or more programming languages (Java mandatory, Scala and Python beneficial) Comfort with software development methodology around unit testing, performance tuning ,integration testing, etc. Experience in real-time data processing Name...

Clear concept of dimensional data modelling(Logical), SCDs, Normalisation and denormalizations. Thoroughly translating the logical models to physical and data flows per the business requirements. Update and optimise the local and metadata models. Could you evaluate the implemented data system for the variances, discrepancies and efficiencies? Troubleshoot and optimise the existing data flows, models and processing jobs by modularising. Explore ways to enhance the data quality and reliability. Strong in writing UDF Previous knowledge of implementing the DQ framework on Spark would be an added advantage. Good in writing programming using Python/Scala/PySpark is a must. Strong knowledge of spar...

6 + years of overall IT experience in Telecom OSS especially in Assurance domain Solution, design, and Implementation - Strong Knowledge of Telecom OSS domain, with excellent experience Service Now for Assuance - Knowledge and experience on Big Data,Data lake solution, KAFKA , Hadoop/Hive. - Experience on Python (pyspark) is essential. - Implementation experience in continuous integration and delivery philosophies and practices specifically on Docker, Git, JenKins - Self driven and highly motivated candidate for a client facing role in a challenging environment

Hands On with advanced SQL, Python etc. Hands On in data profiling Hands On in working on Cloud like Azure and Cloud DW like Databricks Hands On experience on Scheduling tools like Airflow, Control M etc. Knowledgeable on Big Data tools like Spark (python/scala), Hive, Impala, Hue and storage (e.g. HDFS, HBase) Knowledgeable in CICD processes Bitbucket/GitHub, Jenkins, Nexus etc. Knowledgeable managing structured and unstructured data types

8+ years of experience in Life Science / Pharmacy Hands on experience in statistical programming languages like R, Python, Good to have experience in Thingworx analytics and database query languages like SQL, Hive and Pig is desirable. Familiarity with Scala, Java, or C++ is an added advantage. Good applied statistical skills, including knowledge of statistical tests, distributions, regression, maximum likelihood estimators etc. Good knowledge of machine learning methods like k-Nearest neighbors, nave bayes, SVM and decision forests. R OR Python + ML + Regression + classification + clustering + SVM (support vector machine) + decision trees / random forest.

Description Experience in following technologiesHadoop Spark Hive Pig SQL query experience in data application development at scale in the Hadoop ecosystem. Bigdata Spark kafka Java/python/scala Hadoop hive design and architecture problem solving skills Roles and responsibilities Drive the technical design of large scale data platforms utilizing modern and open source technologies. Set standards for data engineering functions; design templates for the data management program which are scalable repeatable and simple Serve as a consultant on all aspects of data engineering analytics and data projects Be able to consult on complex data engineering efforts and lead project teams through the solu...

JD -7+ years of hands on experience in Python especially dealing with Pandas and Numpy Good hands-on experience in Spark PySpark and Spark SQL Hands on experience in Databricks Unity Catalog Delta Lake Lake house Platform Medallion Architecture Azure Data Factory ADLS Experience in dealing with Parquet and JSON file format Knowledge in Snowflake.

Our Purpose Title and Summary Consultant, Advisors & Consulting Services, Performance Analytics Consultant Performance Analytics Advisors & Consulting Services Services within Mastercard is responsible for acquiring, engaging, and retaining customers by managing fraud and risk, enhancing cybersecurity, and improving the digital payments experience. We provide value-added services and leverage expertise, data-driven insights, and execution. Our Advisors & Consulting Services team combines traditional management consulting with Mastercards rich data assets, proprietary platforms, and technologies to provide clients with powerful strategic insights and recommendations. Our teams work with a div...

* 7+ years experience in data eng, data platforms & analytics, 10+ years of consulting experience ? Completed Data Engineering Professional certification & required classes ? Minimum 6-8+ projects delivered with hands-on experience in development on databricks ? Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with deep expertise in at least one ? Deep experience with distributed computing with Spark with knowledge of Spark runtime internals ? Familiarity with CI/CD for production deployments ? Working knowledge of MLOps ? Current knowledge across the breadth of Databricks product and platform features ? Familiarity with optimizations for performance and scalability

4 Years Data Engineering Experience 5 years of consulting experience Completed Data Engineering Professional certification & required classes Minimum 2-3 projects delivered with hands-on experience in Databricks Completed Apache Spark Programming with Databricks, Data Engineering with Databricks, Optimizing Apache Spark on Databricks Experience in Spark and or Hadoop, Flink, Presto, other popular big data engines Familiarity with Databricks multi-hop pipeline architecture

2 Years Data Engineering Experience, 3 years of consulting experience Completed Data Engineering Associate certification & required classes Minimum 1 project delivered with hands-on experience in development on Databricks Completed Apache Spark Programming with Databricks, Data Engineering with Databricks, Optimizing Apache Spark on Databricks SQL delivery experience, and familiarity with Bigquery, Synapse or Redshift Proficient in Python, knowledge of additional databricks programming languages (Scala)

Overview Data Science Team works in developing Machine Learning (ML) and Artificial Intelligence (AI) projects. Specific scope of this role is to develop ML solution in support of ML/AI projects using big analytics toolsets in a CI/CD environment. Analytics toolsets may include DS tools/Spark/Databricks, and other technologies offered by Microsoft Azure or open-source toolsets. This role will also help automate the end-to-end cycle with Azure Pipelines. You will be part of a collaborative interdisciplinary team around data, where you will be responsible of our continuous delivery of statistical/ML models. You will work closely with process owners, product owners and final business users. Thi...

What are we looking for? Seeking for a strong data analyst who can perform independent. Good to have Prior experience in banking domain is plus and understanding the finance various data transactions. Should be strong in the below: Python Pyspark SQL Airflow Trino Hive Offshore Cordination LLD Agile Scrum Linux Openshift Kubernentes Superset Optional skill set: Linux Openshift Kubernentes Superset Same Posting Description for Internal and External Candidates

Key Responsibilities: Develop, configure, and optimize Apache DolphinScheduler workflows for data processing and automation. Design and implement ETL pipelines, job scheduling, and workflow orchestration solutions. Troubleshoot and resolve performance, scalability, and reliability issues in DolphinScheduler. Integrate DolphinScheduler with big data tools such as Hadoop, Spark, Flink, Hive, and Kafka. Work closely with data engineers, DevOps, and software teams to enhance workflow automation. Develop custom plugins and extensions for DolphinScheduler as needed. Monitor and optimize job execution, resource allocation, and workflow efficiency. Maintain best practices for CI/CD, version control,...

Join us as a Domain Architect and be part of an exciting journey in shaping the future of mobile networks! In this role, youll work with cutting-edge LTE and 5G technologies, designing and deploying core network solutions while leveraging cloud-native architectures and automation. Your expertise in telecom protocols, networking, and troubleshooting will be key in delivering seamless solutions to customers. You'll collaborate with both technical and management teams, contribute as an individual and team player, and have opportunities to travel globally. If youre passionate about innovation in mobile networks and cloud technologies, wed love to welcome you to our team! HOW YOU WILL CONTRIBUTE ...

Description: Experience in following technologies: Hadoop Spark Hive Pig SQL query experience in data application development at scale in the Hadoop ecosystem. Bigdata Spark kafka Java/python/scala Hadoop hive design and architecture problem solving skills Roles and responsibilities Drive the technical design of large scale data platforms utilizing modern and open source technologies. Set standards for data engineering functions; design templates for the data management program which are scalable repeatable and simple Serve as a consultant on all aspects of data engineering analytics and data projects Be able to consult on complex data engineering efforts and lead project teams through the s...

Location Chennai and Hyderbad preferred but customer is willing to take resources from Hyderabad Experience 5 to 8 yrs ( U3 ). Exp - 5- 10 Yrs Location - Hyderabad / Chennai Location Proven experience as a development data engineer or similar role, with ETL background. Experience with data integration / ETL best practices and data quality principles. Play a crucial role in ensuring the quality and reliability of the data by designing, implementing, and executing comprehensive testing. By going over the User Stories build the comprehensive code base and business rules for testing and validation of the data. Knowledge of continuous integration and continuous deployment (CI/CD) pipelines. Famil...

Description Detailed JD RTIM Pega CDH 8.8 Multi App Infinity 24.1 Java Restful API oAuth 1. Understanding the NBA requirements and the complete CDH architecture 2. Review of the Conceptual Design Detailed Design and estimations. 3. Reviewing and contributing to the deployment activities and practices 4. Contributing to overall technical solution and putting it to practise. 5. Contributing to the requirement discussion with the Subject matter expertise in relation to Pega CDH 6. Experience in Pega CDH v8.8 multi app or 24.1 and retail banking domain is preferred. 7. Conducting peer code reviews 8. Excellent communications skills Named Job Posting? (if Yes - needs to be approved by SCSC) Addit...

Description Overall 10+ years of experience in Python and Shell Knowledge of distributed systems like Hadoop and Spark as well as cloud computing platforms such as Azure and AWS Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade B Level To Be Defined Named Job Posting? (if Yes - needs to be approved by SCSC) No Remote work possibility No Global Role Family To be defined Local Role Name To be defined Local Skills Ruby;automation;Python Languages RequiredENGLISH Role Rarity To Be Defined

EXP 4 to 6 yrs Location Any PSL Location Rate below 14$ JD - DBT/AWS Glue/Python/Pyspark Hands-on experience in data engineering, with expertise in DBT/AWS Glue/Python/Pyspark. Strong knowledge of data engineering concepts, data pipelines, ETL/ELT processes, and cloud data environments (AWS) Technology DBT, AWS Glue, Athena, SQL, Spark, PySpark Good understanding of Spark internals and how it works. Goot skills in PySpark Good understanding of DBT basically should be to understand DBT limitations and when it will end-up in model explosion Good hands-on experience in AWS Glue AWS expertise should know different services and should know how to configure them and infra-as-code experience Basic ...

Developer leads the cloud application development/deployment. A developer responsibility is to lead the execution of a project by working with a senior level resource on assigned development/deployment activities and design, build, and maintain cloud environments focusing on uptime, access, control, and network security using automation and configuration management tools Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Strong proficiency in Java, Spring Framework, Spring boot, RESTful APIs, excellent understanding of OOP, Design Patterns. Strong knowledge of ORM tools like Hibernate or JPA, Java based Micro-services framew...