756 Apache Pig Jobs - Page 4

6-7+ years of Experience in Hadoop or any Cloud Bigdata components (specific to the Data Engineering role), Hadoop hands on (airflow, oozie, hive, hdfs, sqoop, pig, map, reduce ) 4+ years of Exp in Spark (Spark Batch, Streaming, Mlibetc). Candidates should possess proficiency in utilizing the Apache Spark framework. 6-7+ years of experience in Python programming language. 4+ years of experience with pyspark data transformation(json,csv,rdbms,stream) pipeline design , development and deployment with kubernate/onprem platform (not cloud based). 2+ years of experience in designing and implementing data workflows with Apache Airflow. Kafka or equivalent Cloud Bigdata components (specific to the ...

About The Role Project Role : Infra Tech Support Practitioner Project Role Description : Provide ongoing technical support and maintenance of production and development systems and software products (both remote and onsite) and for configured services running on various platforms (operating within a defined operating model and processes). Provide hardware/software support and implement technology at the operating system-level across all server and network areas, and for particular software solutions/vendors/brands. Work includes L1 and L2/ basic and intermediate level troubleshooting. Must have skills : Apache Cassandra Good to have skills : NAMinimum 3 year(s) of experience is required Educ...

Role Purpose The purpose of this role is to design, develop and troubleshoot solutions/ designs/ models/ simulations on various softwares as per clients/ project requirements Do 1. Design and Develop solutions as per clients specifications Work on different softwares like CAD, CAE to develop appropriate models as per the project plan/ customer requirements Test the protype and designs produced on the softwares and check all the boundary conditions (impact analysis, stress analysis etc) Produce specifications and determine operational feasibility by integrating software components into a fully functional software system Create a prototype as per the engineering drawings & outline CAD model is...

Location Hyd / Pune / Bengaluru JD 1ETL, Pyspark Experience4 8 years Proven experience as a development data engineer or similar role, with ETL background. Experience with data integration / ETL best practices and data quality principles. Play a crucial role in ensuring the quality and reliability of the data by designing, implementing, and executing comprehensive testing. By going over the User Stories build the comprehensive code base and business rules for testing and validation of the data. Knowledge of continuous integration and continuous deployment (CI/CD) pipelines. Familiarity with Agile/Scrum development methodologies. Excellent analytical and problem solving skills. Strong communi...

JR REQ---Data Engineer(Pyspark, Big mailto:data)--4to8year---hyd----hemanth.karanam@tcs.com----TCS C2H ---900000

Data engineer Skills Required: Strong proficiency in Pyspark , Scala , and Python Experience with AWS Glue Experience Required: Minimum 5 years of relevant experience Location: Available across all UST locations Notice Period: Immediate joiners (Candidates available to join by 31st January 2025 ) SO - 22978624 Location - Chandigarh,Dadra & Nagar Haveli,Daman,Diu,Goa,Haveli,Jammu,Lakshadweep,Nagar,New Delhi,Puducherry,Sikkim

JR REQ -BigData Engineer --4to8year---HYD----Karuppiah Mg --- TCS C2H ---900000

Client expectation apart from JD Longer AWS data engineering experience (glue, spark, ECR ECS docker), python, pyspark, hudi/iceberg/Terraform, Kafka. Java in early career would be a great addition but not a prio. (for the OOP part and java connectors).

Roles and Responsibilities Client Impact Lead client engagements across a range of industries and problem statements Develop analytics strategies and programs for large, regional, and global clients by leveraging data and technology solutions to unlock client value Team Collaboration & Culture Lead team to creative insights and sound business recommendations, and deliver impactful client presentations while growing team members roles and skills Provide analytical and day-to-day project delivery team leadership, and create a collaborative and inclusive environment for all levels Collaborate with internal Mastercard stakeholders including Product and Business Development to scope projects, cre...

Educational Requirements Bachelor of Engineering Service Line Data & Analytics Unit Responsibilities A day in the life of an Infoscion As part of the Infosys delivery team, your primary role would be to interface with the client for quality assurance, issue resolution and ensuring high customer satisfaction. You will understand requirements, create and review designs, validate the architecture and ensure high levels of service offerings to clients in the technology domain. You will participate in project estimation, provide inputs for solution delivery, conduct technical risk planning, perform code reviews and unit test plan reviews. You will lead and guide your teams towards developing opti...

Position Summary This Requisition is for the Employee Referrals Campaign This is a Leadership role for a person who will be responsible for leading and managing a team of Managers and senior managers working on multiple project streams for a client. They would be applying domain knowledge for better business decision-making, especially in the areas of Analytics/Technology/Product Engineering/Product Management. Job Responsibilities Leadership - Lead & guide the team independently or with little support to implement & deliver complex project assignments. Logical Thinking - Able to think analytically, use a systematic and logical approach to analyze data, problems, and situations. Must be able...

NEX Softsys is looking for Big Data Developer to join our dynamic team and embark on a rewarding career journey Design, develop, and maintain big data solutions to meet business requirements and support data-driven decision making Work with stakeholders to understand their data needs and determine how to best use big data technologies to meet those needs Design and implement scalable, high-performance big data architectures, using technologies such as Hadoop, Spark, and NoSQL databases Extract, transform, and load large data sets into a big data platform for analysis and reporting Write complex SQL queries and develop custom scripts to process big data Collaborate with data scientists, data ...

Role Description : As a Senior Cloud Data Platform (GCP) Engineer at Incedo, you will be responsible for managing and optimizing the Google Cloud Platform environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or Azure and have experience with big data technologies such as Hadoop or Spark. You will be responsible for configuring and optimizing the GCP environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security tea...

Role Description As a Senior Cloud Data Platform (GCP) Engineer at Incedo, you will be responsible for managing and optimizing the Google Cloud Platform environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or Azure and have experience with big data technologies such as Hadoop or Spark. You will be responsible for configuring and optimizing the GCP environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security team ...

Hive/Python/Spark Technical hands on data processing Database SQL knowledge for retrieval of data - transformation queries such as joins (full, left , right) , ranking , group by Database SQL/Oracle Good Communication skills. Additional skills - GitHub , Jenkins , shell scripting would be added advantage, large-scale data processing model implementation, and performance tracking Mandatory Competencies Big Data - Big Data - HIVE Big Data - Big Data - Pyspark Big Data - Big Data - SPARK Data Science and Machine Learning - Data Science and Machine Learning - Data Analyst Database - Oracle - PL/SQL Packages Programming Language - Python - Python Shell Beh - Communication and collaboration

Clear concept of dimensional data modelling(Logical), SCDs, Normalisation and denormalizations. Thoroughly translating the logical models to physical and data flows per the business requirements. Update and optimise the local and metadata models. Could you evaluate the implemented data system for the variances, discrepancies and efficiencies? Troubleshoot and optimise the existing data flows, models and processing jobs by modularising. Explore ways to enhance the data quality and reliability. Strong in writing UDF Previous knowledge of implementing the DQ framework on Spark would be an added advantage. Good in writing programming using Python/Scala/PySpark is a must. Strong knowledge of spar...

6 + years of overall IT experience in Telecom OSS especially in Assurance domain Solution, design, and Implementation - Strong Knowledge of Telecom OSS domain, with excellent experience Service Now for Assuance - Knowledge and experience on Big Data,Data lake solution, KAFKA , Hadoop/Hive. - Experience on Python (pyspark) is essential. - Implementation experience in continuous integration and delivery philosophies and practices specifically on Docker, Git, JenKins - Self driven and highly motivated candidate for a client facing role in a challenging environment

Hands On with advanced SQL, Python etc. Hands On in data profiling Hands On in working on Cloud like Azure and Cloud DW like Databricks Hands On experience on Scheduling tools like Airflow, Control M etc. Knowledgeable on Big Data tools like Spark (python/scala), Hive, Impala, Hue and storage (e.g. HDFS, HBase) Knowledgeable in CICD processes Bitbucket/GitHub, Jenkins, Nexus etc. Knowledgeable managing structured and unstructured data types

8+ years of experience in Life Science / Pharmacy Hands on experience in statistical programming languages like R, Python, Good to have experience in Thingworx analytics and database query languages like SQL, Hive and Pig is desirable. Familiarity with Scala, Java, or C++ is an added advantage. Good applied statistical skills, including knowledge of statistical tests, distributions, regression, maximum likelihood estimators etc. Good knowledge of machine learning methods like k-Nearest neighbors, nave bayes, SVM and decision forests. R OR Python + ML + Regression + classification + clustering + SVM (support vector machine) + decision trees / random forest.

Description Experience in following technologiesHadoop Spark Hive Pig SQL query experience in data application development at scale in the Hadoop ecosystem. Bigdata Spark kafka Java/python/scala Hadoop hive design and architecture problem solving skills Roles and responsibilities Drive the technical design of large scale data platforms utilizing modern and open source technologies. Set standards for data engineering functions; design templates for the data management program which are scalable repeatable and simple Serve as a consultant on all aspects of data engineering analytics and data projects Be able to consult on complex data engineering efforts and lead project teams through the solu...

JD -7+ years of hands on experience in Python especially dealing with Pandas and Numpy Good hands-on experience in Spark PySpark and Spark SQL Hands on experience in Databricks Unity Catalog Delta Lake Lake house Platform Medallion Architecture Azure Data Factory ADLS Experience in dealing with Parquet and JSON file format Knowledge in Snowflake.

About the Team At Nutanix, you would be joining the Next Gen- Data Engineering team comprised of more than 40 skilled professionals. Our team thrives in an open culture with a flat hierarchy, where collaboration and innovation are at the heart of everything we do. You can expect to work alongside talented individuals who are passionate about pioneering new solutions in the tech industry. You will report to the Senior Engineering Manager, who is dedicated to providing guidance and support for your career growth. The work setup at Nutanix is a hybrid model, with employees expected to be in the office 2 days a week. This structure allows for both in-person collaboration and flexibility for remo...

We are seeking a highly skilled Senior Engineer Consultant in Mobile Engineering. The successful candidate will be responsible for designing, implementing, and managing mobile device management (MDM) solutions using Intune and Azure. This person will create and maintain secure and scalable solutions for managing mobile devices, applications, and data. The Engineer will stay current with industry trends and emerging technologies to develop solutions that support and grow our business. Primary Responsibilities: Configure and deploy policies, profiles, compliance settings, and app protection policies for mobile devices and applications Monitor and troubleshoot device and app issues, as well as ...

We are looking for a highly motivated and detail-oriented Catastrophe Data Analyst to join our team at Swiss Re. The ideal candidate should have 0 to 7 years of experience in the field. Roles and Responsibility Analyze and interpret catastrophe data to identify trends and patterns. Develop and maintain databases for storing and managing catastrophe data. Collaborate with cross-functional teams to design and implement new catastrophe models. Conduct research and stay updated on industry developments and advancements in catastrophe modeling. Provide insights and recommendations to stakeholders based on analysis results. Ensure data quality and integrity by implementing data validation and veri...

We are seeking a talented and experienced Databricks Engineer to join our innovative team in Bengaluru, India. In this role, you will be responsible for designing, implementing, and optimising data pipelines and analytics solutions using the Databricks platform. You will collaborate closely with cross-functional teams to deliver high-quality, scalable data solutions that drive business insights and decision-making. Develop and optimise ETL processes using Databricks and Apache Spark Design and implement efficient data models and schemas for optimal data storage and retrieval Collaborate with data scientists and analysts to build and deploy machine learning models Ensure data quality, consist...