654 Apache Pig Jobs - Page 12

Looking for a skilled Big Data Engineer to join our team at Hexaware Technologies Ltd. The ideal candidate will have experience in the IT Services & Consulting industry and possess skills in big data engineering. Roles and Responsibility Design, develop, and implement big data solutions using various technologies. Collaborate with cross-functional teams to identify business requirements and develop technical solutions. Develop and maintain large-scale data pipelines and architectures. Ensure data quality, integrity, and security. Optimize big data systems for performance and scalability. Stay updated with industry trends and emerging technologies in big data. Job Requirements Strong understa...

We are seeking a highly skilled Data Engineer with expertise in ETL techniques, programming, and big data technologies. The candidate will play a critical role in designing, developing, and maintaining robust data pipelines, ensuring data accuracy, consistency, and accessibility. This role involves collaboration with cross-functional teams to enrich and maintain a central data repository for advanced analytics and machine learning. The ideal candidate should have experience with cloud-based data platforms, data modeling, and data governance processes. Location - Bengaluru,Doddakannell, Karnataka, Sarjapur Road

We are seeking a skilled Big Data Developer with 3+ years of experience to develop, maintain, and optimize large-scale data pipelines using frameworks like Spark, PySpark, and Airflow. The role involves working with SQL, Impala, Hive, and PL/SQL for advanced data transformations and analytics, designing scalable data storage systems, and integrating structured and unstructured data using tools like Sqoop. The ideal candidate will collaborate with cross-functional teams to implement data warehousing strategies and leverage BI tools for insights. Proficiency in Python programming, workflow orchestration with Airflow, and Unix/Linux environments is essential. Location: Remote- Delhi / NCR,Banga...

We are seeking a skilled Big Data Developer with 3+ years of experience to develop, maintain, and optimize large-scale data pipelines using frameworks like Spark, PySpark, and Airflow. The role involves working with SQL, Impala, Hive, and PL/SQL for advanced data transformations and analytics, designing scalable data storage systems, and integrating structured and unstructured data using tools like Sqoop. The ideal candidate will collaborate with cross-functional teams to implement data warehousing strategies and leverage BI tools for insights. Proficiency in Python programming, workflow orchestration with Airflow, and Unix/Linux environments is essential. Locations : Mumbai, Delhi / NCR, Be...

Job TitleOpenShift Architect LocationIndia (remote) Job TypeFull-Time Shift Timing2:30 PM IST to 11:30 PM IST Company Overview: At Codvo, software and people transformations go"hand-in-hand We are a global empathy-led technology services company Product innovation and mature software engineering are part of our core DNA Respect, Fairness, Growth, Agility, and Inclusiveness are the core values that we aspire to live by each day We continue to expand our digital strategy, design, architecture, and product management capabilities to offer expertise, outside-the-box thinking, and measurable results About The Role : We are seeking a seasoned Red Hat OpenShift Architect to join our team and play a...

Developer leads the cloud application development/deployment. A developer responsibility is to lead the execution of a project by working with a senior level resource on assigned development/deployment activities and design, build, and maintain cloud environments focusing on uptime, access, control, and network security using automation and configuration management tools Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Strong proficiency in Java, Spring Framework, Spring boot, RESTful APIs, excellent understanding of OOP, Design Patterns. Strong knowledge of ORM tools like Hibernate or JPA, Java based Micro-services framew...

Developer leads the cloud application development/deployment. A developer responsibility is to lead the execution of a project by working with a senior level resource on assigned development/deployment activities and design, build, and maintain cloud environments focusing on uptime, access, control, and network security using automation and configuration management tools Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Strong proficiency in Java, Spring Framework, Spring boot, RESTful APIs, excellent understanding of OOP, Design Patterns. Strong knowledge of ORM tools like Hibernate or JPA, Java based Micro-services framew...

Developer leads the cloud application development/deployment. A developer responsibility is to lead the execution of a project by working with a senior level resource on assigned development/deployment activities and design, build, and maintain cloud environments focusing on uptime, access, control, and network security using automation and configuration management tools. Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Strong proficiency in Java, Spring Framework, Spring boot, RESTful APIs, excellent understanding of OOP, Design Patterns. Strong knowledge of ORM tools like Hibernate or JPA, Java based Micro-services frame...

Experience Level: 10+ years Required Skills and Expertise: Experience: 10+ years of experience in data engineering, data architecture, data science and reporting. Data Architecture: Proven expertise in designing and managing scalable data architecture, including data lakes, warehouses, and pipelines. Data Science: Strong understanding of machine learning, statistical modeling, and data analytics. Power BI: Understanding the dashboard, reports and underline semantic layer and queries Mongo: Understanding and usage of mongo for troubleshooting, development perspective Kubernetes/Dockers: Current env is in containers based so it is nice to have Python Expertise: Hands-on and advanced proficienc...

Azure databricks- Build Data Pipelines using ADB-Leveraging Python and Spark Build Data Transformation in ADB Convert Scala Pgms to Py Spark Pgms Understand the SCALA program in DataBricks ADF Secondary Skills: Azure data lake gen 2 Synapse analytics (dedicated SQL pool) Exp in delta Lake Pyspark Nice to Have : Tidal Scheduler , Exp in Healthcare Domain

ADB- DE: Primary: Azure, Databricks,ADF, Pyspark/Python Secondary:Datawarehouse, Must Have 10+ Years of IT experience in Datawarehouse and ETL Hands-on data experience on Cloud Technologies on Azure, ADF, Synapse, Pyspark/Python Ability to understand Design, Source to target mapping (STTM) and create specifications documents Flexibility to operate from client office locations Able to mentor and guide junior resources, as needed Nice to Have Any relevant certifications Banking experience on RISK & Regulatory OR Commercial OR Credit Cards/Retail"

You should have: Experience with cloud database platforms, e.g. Azure SQL Database, Snowflake etc, or on-prem database platforms like MS SQL server, and have good SQL skills Experience working with Azure Data Factory (ADF). Experience with data modelling. Experience of working with a variety of stakeholders including product owners, delivery managers and architects. Experience with performance tuning and relational database design particularly in the development of business intelligence solutions. Experience with data migration strategies from on premise to cloud. Experience with Data Warehouse concepts. Experience of communicating and documenting technical design proposals. Experience with ...

Strong Experience in Apache Hadoop, Spark, Spark SQL, Hive, Impala, Yarn, Talend, Hue Strong Experience in SQL queries/stored Procedures/Functions/Triggers, work experience in banking projects focused on liquidity management Spark Calculators based on business logic/rules Programming and reverse engineering skills with Scala, Java, Python, SQL, Unix/Linux shell, Spark RD Understanding of OOP and Functional design approach. Big Data Reporting, Querying and analysis Create T SQL queries/stored Procedures/Functions/Triggers using SQL or Server 2014 and 2017 or any other any other query language Should have fair ETL knowledge. Strong writing, communication, time-management, decision-making, and ...

Primary skills: Snowflake, SQL Secondary skills; Any ETL, Data modeling Around 7+ years of experience in snowflake Excellent in DQH concepts Designing data pipelines: Data engineers design, build, and maintain data pipelines to ingest data from sources like databases, cloud storage, or streaming platforms Good experience in Data models Building data architecture: Data engineers plan, create, and maintain data architectures that are aligned with business needs Collecting data: Data engineers obtain data from suitable sources before working on the database Identifying patterns and creating models: Data engineers use descriptive data models to extract historically relevant insight from data Dat...

About The Role Project Role : Engineering Services Practitioner Project Role Description : Assist with end-to-end engineering services to develop technical engineering solutions to solve problems and achieve business objectives. Solve engineering problems and achieve business objectives using scientific, socio-economic, technical knowledge and practical experience. Work across structural and stress design, qualification, configuration and technical management. Must have skills : 5G Wireless Networks & Technologies Good to have skills : AWS Core Infrastructure, Red Hat Cloud Architecture and DesignMinimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time edu...

Position Purpose The Senior Spark Developer will be responsible for integrating and maintaining the Quantexa platform, a spark based software provided by a UK fintech, into our existing systems to enhance our anti-money laundering capabilities. This role requires a deep expertise in Spark development, as well as an ability to analyze and understand underlying data. Additionally, the candidate should have an interest in exploring open-source applications distributed by Apache, Kubernetes, OpenSearch and Oracle. Responsibilities Direct Responsibilities Integrate and upgrade the Quantexa tool with our existing systems for enhanced anti-money laundering measures. Develop and maintain Spark-based...

Excellent Knowledge on Spark; The professional must have a thorough understanding Spark framework, Performance Tuning etc Excellent Knowledge and hands-on experience of at least 4+ years in Scala and PySpark Excellent Knowledge of the Hadoop eco System- Knowledge of Hive mandatory Strong Unix and Shell Scripting Skills Excellent Inter-personal skills and for experienced candidates Excellent leadership skills Mandatory for anyone to have Good knowledge of any of the CSPs like Azure,AWS or GCP; Certifications on Azure will be additional Plus. Mandatory Skills: PySpark. Experience: 5-8 Years.

Long Description Experienceand Expertise inany of the followingLanguagesat least 1 of them : Java, Scala, Python Experienceand expertise in SPARKArchitecture Experience in the range of 6-10 yrs plus Good Problem SolvingandAnalytical Skills Ability to Comprehend the Business requirementand translate to the Technical requirements Good communicationand collaborative skills with fellow teamandacross Vendors Familiar with development of life cycle includingCI/CD pipelines. Proven experienceand interested in supportingexistingstrategicapplications Familiarity workingwithagile methodology Mandatory Skills: Scala programming.: Experience: 5-8 Years.

The candidate must possess in-depth functional knowledge of the process area and apply it to operational scenarios to provide effective solutions. He/she must be able to identify discrepancies and propose optimal solutions by using a logical, systematic, and sequential methodology. It is vital to be open-minded towards inputs and views from team members and to effectively lead, control, and motivate groups towards company objects. Additionally, he/she must be self-directed, proactive, and seize every opportunity to meet internal and external customer needs and achieve customer satisfaction by effectively auditing processes, implementing best practices and process improvements, and utilizing ...

CostMasters is looking for Internship to join our dynamic team and embark on a rewarding career journeyinvalid title Support the team with daily tasks, research, and documentation while gaining practical exposure to industry practices Assist in project execution, data collection, and report preparation under supervision Participate in meetings, training sessions, and brainstorming activities to enhance learning and contribute fresh ideas Ensure timely completion of assigned tasks with accuracy and professionalism Collaborate with colleagues, adapt to feedback, and uphold organizational policies and standards Demonstrate eagerness to learn, develop new skills, and build a foundation for futur...

CostMasters is looking for Data Engineer to join our dynamic team and embark on a rewarding career journey Liaising with coworkers and clients to elucidate the requirements for each task. Conceptualizing and generating infrastructure that allows big data to be accessed and analyzed. Reformulating existing frameworks to optimize their functioning. Testing such structures to ensure that they are fit for use. Preparing raw data for manipulation by data scientists. Detecting and correcting errors in your work. Ensuring that your work remains backed up and readily accessible to relevant coworkers. Remaining up-to-date with industry standards and technological advancements that will improve the qu...

DIATOZ SOLUTIONS PVT LTD is looking for Data Engineer Apache NiFi & Groovy to join our dynamic team and embark on a rewarding career journey Liaising with coworkers and clients to elucidate the requirements for each task. Conceptualizing and generating infrastructure that allows big data to be accessed and analyzed. Reformulating existing frameworks to optimize their functioning. Testing such structures to ensure that they are fit for use. Preparing raw data for manipulation by data scientists. Detecting and correcting errors in your work. Ensuring that your work remains backed up and readily accessible to relevant coworkers. Remaining up-to-date with industry standards and technological adv...

Myridius x Aethereus is looking for Data Engineer to join our dynamic team and embark on a rewarding career journey Liaising with coworkers and clients to elucidate the requirements for each task. Conceptualizing and generating infrastructure that allows big data to be accessed and analyzed. Reformulating existing frameworks to optimize their functioning. Testing such structures to ensure that they are fit for use. Preparing raw data for manipulation by data scientists. Detecting and correcting errors in your work. Ensuring that your work remains backed up and readily accessible to relevant coworkers. Remaining up-to-date with industry standards and technological advancements that will impro...

We are seeking a skilled Databricks Engineer to join our team in Bengaluru, India. In this role, you will be responsible for designing, implementing, and maintaining data pipelines and analytics solutions using the Databricks platform. You will work closely with cross-functional teams to deliver high-quality, scalable data solutions that drive business insights and decision-making. Develop and optimize ETL processes using Databricks and Apache Spark Design and implement data models and schemas for efficient data storage and retrieval Collaborate with data scientists and analysts to build and deploy machine learning models Ensure data quality, consistency, and reliability across all data pipe...

Developer leads the cloud application development/deployment. A developer responsibility is to lead the execution of a project by working with a senior level resource on assigned development/deployment activities and design, build, and maintain cloud environments focusing on uptime, access, control, and network security using automation and configuration management tools Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Primary Skills: Core Java, Spring Boot, Java2/EE, Microsservices Hadoop Ecosystem (HBase, Hive, MapReduce, HDFS, Pig, Sqoop etc) Spark Good to have Python Preferred technical and professional experience None