22039 Hadoop Jobs - Page 35

We are looking for a passionate and detail-oriented Software Engineer to join our Big Data / Data Warehouse (DWH) / ETL team. The ideal candidate should have a strong understanding of data systems, data processing pipelines, and an interest in large-scale data analytics. Fresh graduates with relevant academic projects or training in data engineering are also welcome to apply. Responsibilities Design, develop, and maintain ETL pipelines for efficient data extraction, transformation, and loading. Work with Big Data technologies (Hadoop, Spark, Hive, etc. ) to process and analyse large datasets. Develop and maintain data warehouse models and ensure data integrity and consistency. Optimise and t...

Immediate opening for " Python & Big Data Engineering" Job Title: Python & Big Data Engineer Job Location: Chennai (Hybrid) Experience Required: 4 -9 Years Job Requirement: Strong proficiency in Python with OOP and functional programming experience. Experience building multi-module, package-structured applications . Expertise in unit testing using pytest / unittest. Experience managing environments using venv / conda / pyenv . Proficient in SQL for complex data querying & performance optimization. Hands-on experience with Teradata for data extraction & transformation. (Highly Preferred) Working knowledge of Hadoop ecosystem components. PySpark / Apache Spark experience for distributed data p...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics - D and A – AWS DBX - Staff As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufacturing, Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for Managers (GT...

Job Description Alimentation Couche-Tard Inc., (ACT) is a global Fortune 200 company. A leader in the convenience store and fuel space, it has footprint across 31 countries and territories. The India Data & Analytics Global Capability Centre is an integral part of ACT’s Global Data & Analytics Team and the Associate Data Scientist will be a key player on this team that will help grow analytics globally at ACT. About The Role The hired candidate will partner with multiple departments, including Global Marketing, Merchandising, Global Technology, and Business Units. The incumbent will be responsible for delivering advanced analytics projects that drive business results including interpreting b...

Job Description Alimentation Couche-Tard Inc., (ACT) is a global Fortune 200 company. A leader in the convenience store and fuel space, it has footprint across 31 countries and territories. The India Data & Analytics Global Capability Centre is an integral part of ACT’s Global Data & Analytics Team and the Associate Data Scientist will be a key player on this team that will help grow analytics globally at ACT. About The Role The hired candidate will partner with multiple departments, including Global Marketing, Merchandising, Global Technology, and Business Units. The incumbent will be responsible for delivering advanced analytics projects that drive business results including interpreting b...

Experience : 12.00 + years Salary : Confidential (based on experience) Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Office (Noida) Placement Type : Full time Permanent Position (*Note: This is a requirement for one of Uplers' client - Nuaav) What do you need for this opportunity? Must have skills required: Synapse Analytics, Azure SQL Database, Azure Data Factory, Spark, Hadoop, Azure Data Lake, CI - CD, ETL, CI/CD Nuaav is Looking for: Job Description: Company Overview Nuaav is a boutique consulting firm that positions itself as a strategic partner for corporate leaders, focusing on delivering real business results and driving impactful change in complex and rapidly evolving bu...

As AWS Data Engineer at organization, you will play a crucial role in the design, development, and maintenance of our data infrastructure. Your work will empower data-driven decision-making and contribute to the success of our data-driven initiatives You will design and maintain scalable data pipelines using AWS data analytical resources, enabling efficient data processing and analytics. Key Responsibilities: - Highly experiences in developing ETL pipelines using AWS Glue and EMR with PySpark/Scala. - Utilize AWS services (S3, Glue, Lambda, EMR, Step Functions) for data solutions. - Design scalable data models for analytics and reporting. - Implement data validation, quality, and governance ...

Description: Job Discription:Employee must have experince in Conflunet Kafka Configuration and orchestration of containers using Kubernetes/OpenShift as an orchestration tool. Administration of infrastructure for Tier1 applications In-depth knowledge in Kafka administration, with skills to manage Kafka clusters in production environments. Release Management, Jenkins, Git, Ansible, Linux, AWS/Azure/GCP cloud, Docker, Kubernetes/OpenShift, Tomcat, and shell scripting. Good knowledge of virtualization and container technology like Docker. Experience working with various orchestration platforms both on On-Premise and cloud servers. Designing and deploying applications utilizing Azure/GCP/AWS sta...

Description: Proficiency/good understanding in at least couple of the following:Ab Initio SQL Data Modelling / relationships RDBMS Hadoop Machine Learning algorithms Ensure consistency in approach design and output to deliver quality and concise design solution documentation. Lead best practice methodology for design Define promote re-usable extendible scalable maintainable solution considering tradeoff for cost vs benefit Communicate at all levels in a clear and credible way about the importance of solution design. Lead developers in the team and foster a shared vision for design. Knowledgeable and skilled at the practice of data driven design solutions. Expert communications to all levels ...

Description: Proficiency/good understanding in at least couple of the following:Ab Initio SQL Data Modelling / relationships RDBMS Hadoop Machine Learning algorithms Ensure consistency in approach design and output to deliver quality and concise design solution documentation. Lead best practice methodology for design Define promote re-usable extendible scalable maintainable solution considering tradeoff for cost vs benefit Communicate at all levels in a clear and credible way about the importance of solution design. Lead developers in the team and foster a shared vision for design. Knowledgeable and skilled at the practice of data driven design solutions. Expert communications to all levels ...

Description: Objective:Build and maintain data pipelines and infrastructure to support data storage and processing. Key Responsibilities: -Design develop and optimize data pipelines using Azure Databricks. -Ensure data reliability scalability and quality. -Integrate data sources into a unified platform. -Monitor and troubleshoot data workflows. Skills Required: -Expertise in Azure Databricks Spark and big data tools. -Strong programming skills (e.g. Python Scala SQL). -Knowledge of ETL processes and cloud technologies. -Familiarity with CI/CD practices for data pipelines Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade :C Level :To Be Defined Named J...

Enhance, optimize and maintain existing data ingestion, transformation and extraction pipelines and assets built for reporting and analytics on Cloud (GCP + BigQuery + Confluent Cloud) and Big Data(Cloudera and Confluent Platform) platforms. Work with the Product Owner to understand the priorities and OKRs for the quarter and gather detailed requirements from the initiative owners or program sponsor as per the Epics planned to be delivered in the quarter. Build new and optimized data pipelines, assets to meet the end user requirements. The Data pipelines must adhere to all the architecture, design and engineering principles. Design the data pipelines and assets to meet non functional require...

Description: 1. Job Summary:We are seeking a C2 Level skilled HVR Administrator to join our team for a critical data migration project at GE Vernova. The successful candidate will be responsible for overseeing the migration of data from GE's OneStream ERP system to a new data lake using HVR (High Volume Replication). This role involves the initial installation and configuration of HVR on a new instance, ensuring seamless data replication and integration. 2.Key Responsibilities:HVR Installation and Configuration:Install and configure HVR on the new instance. Set up HVR hub, channels, and locations for data replication. Data Migration:Plan and execute the migration of data from OneStream ERP t...

Description: C1 and up Generic JD: Be part of Global production support team supporting health and compliance of Technology Infrastructure systems including but not limited to, UNIX and Windows Servers, Storage (NAS/SAN), backups and Databases Perform support tasks as documented and aligned to each technology platform supported. This includes but is not limited to:UNIX, Linux, Windows Server, NAS, SAN, Backups, DB2, Hadoop, Sybase, SQL Ensuring critical issues are attended in a timely manner and escalated when appropriate Prioritizing tasks and adhering to Service Level Targets Contribute in Root Cause Analysiss What you Bring: Required: Relevant Experience in managing Linux Solaris and Wind...

Description: C1 and up JD: Be part of Global production support team supporting health and compliance of Technology Infrastructure systems including but not limited to, UNIX and Windows Servers, Storage (NAS/SAN), backups and Databases Perform support tasks as documented and aligned to each technology platform supported. This includes but is not limited to:UNIX, Linux, Windows Server, NAS, SAN, Backups, DB2, Hadoop, Sybase, SQL Ensuring critical issues are attended in a timely manner and escalated when appropriate Prioritizing tasks and adhering to Service Level Targets Contribute in Root Cause Analysiss What you Bring: Required: Relevant Experience in managing Linux Solaris and Windows Serv...

Description: C1 and up Generic JD: Be part of Global production support team supporting health and compliance of Technology Infrastructure systems including but not limited to, UNIX and Windows Servers, Storage (NAS/SAN), backups and Databases Perform support tasks as documented and aligned to each technology platform supported. This includes but is not limited to:UNIX, Linux, Windows Server, NAS, SAN, Backups, DB2, Hadoop, Sybase, SQL Ensuring critical issues are attended in a timely manner and escalated when appropriate Prioritizing tasks and adhering to Service Level Targets Contribute in Root Cause Analysiss What you Bring: Required: Relevant Experience in managing Linux Solaris and Wind...

Description Primary Skill Kafka Cluster Management,Kafka admin,Kubernetes,Helm ,DevOps, Jenkins Secondary Skill:Grafana, Prometheus, Dynatrace Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade C Level To Be Defined Named Job Posting? (if Yes - needs to be approved by SCSC) No Remote work possibility No Global Role Family To be defined Local Role Name To be defined Local Skills Kafka Cluster Management;Kubernetes Languages RequiredENGLISH Role Rarity To Be Defined

Description Job Summary Role Value Proposition MetLife Data & Analytics organization is the team of expert technologists responsible for building big data platforms and data services with innovative technologies to enable MetLife businesses to generate insights and value to its customers. The team is the center of excellence in data engineering in MetLife and plays a key role in data enablement through multiple data stores supporting different kind of analytical use cases to be able to derive predictive prescriptive and descriptive insights. The Azure Data Engineer III serves as big data development expert within the data analytics engineering organization of MetLife Data & Analytics. This p...

Description Key Responsibilities Analyse raw data from various source files (e.g. EXCEL CSV JSON XML Parquet etc.) to identify formatting issues inconsistencies missing values and discrepancies. Use PySpark to transform clean and restructure source data to match the expected format for further processing and analysis. Apply data validation rules and data quality checks to correct issues related to data integrity including handling null values duplicate records and data type mismatches. Develop PySpark jobs to transform raw data into standardized formats (e.g. converting date formats normalizing text fields correcting encoding issues). Ensure that data transformations meet the required schema...

Location Hyd / Pune / Bengaluru JD 1ETL, Pyspark Experience4 8 years Proven experience as a development data engineer or similar role, with ETL background. Experience with data integration / ETL best practices and data quality principles. Play a crucial role in ensuring the quality and reliability of the data by designing, implementing, and executing comprehensive testing. By going over the User Stories build the comprehensive code base and business rules for testing and validation of the data. Knowledge of continuous integration and continuous deployment (CI/CD) pipelines. Familiarity with Agile/Scrum development methodologies. Excellent analytical and problem solving skills. Strong communi...

Title / Role No of Pos About The Role Must Skills Yrs of Exp Location Work Timings Scala+spark scala+spark/play+hive+cicd+cloud(aws perfered or any other cloud) and sql, candidate should be good with advanced programming language (scala) Scala,(either spark or play),sql & cloud(aws preferred) 5+yr Hyderabad WFO

Description Key job responsibilities include, but are not limited to Become involved with day-to-day operations in Production Control. Understand the workflows and be able to troubleshoot issues and define new workflows in PCS and Control-M Design Airflow infrastructure according to the industry best practices. Work with System Administrators, Platform Engineers, Network, Firewall and Monitoring teams on the implementation of the infrastructure. Document the system's specifics, define, and implement administrative controls and policies of Airflow. Work with the development teams to identify the groups of jobs to migrate to airflow. Assist the development teams with the migrationsdefine DAGs,...

Description The position involves working within our CX Program Implementation team with a focus on building the surveys and reporting. Your day would include working with the business analyst to understand CX requirements shared by the stakeholders and work with the implementation team to develop solutions. The role requires a creative thinker who can solve problems through innovative technology solutions while supporting the development of customer experience programs. Responsibilities ? Configure the Medallia surveys and reporting setup as per the requirements shared by the Business Analyst. ? Build Auto Importers for the new survey programs. ? Conduct file inflow reviews to understand th...

Data ModelingThey create data models using Entity Relationship (ER) diagrams to define how data entities interact and relate to each other. This ensures data organization and accessibility. Data Architecture DesignThey design and implement the technical architecture for data storage, processing, and retrieval. This includes data lakes, data warehouses, and data pipelines. Data GovernanceData architects establish data governance policies and procedures to ensure data quality, security, and compliance with regulations Skills: Strong understanding of data modeling concepts (ER diagrams) Knowledge of database technologies (SQL, NoSQL) Expertise in data warehousing and data lakes Experience with ...

Data Governance The ideal candidate for this position should have worked on data governance projectsbefore. It would be beneficial if they are familiar with tools like IBM IGC and Orion Datagovernance tool. Knowing Nordeas data platforms and models is a plus. This job is for asenior or expert IT analyst, so we expect them to cllaborate with data architects andbusiness consultants to outline IT needs. They will also lead important meetings andworkshops, create documents that outline technology plan to share with Senior Leaders.Description of knowledge and experienceA minimum of 6 years experience in the IT industry, with a focus on IT analysis andrequirements management. Expertise in Data Gov...