1460 Mapreduce Jobs - Page 13

We are looking for a skilled Big Data Engineer with 6 to 22 years of experience. The ideal candidate will have expertise in Pyspark, Azure Data Bricks, and workflows. This position is available as a contract role across Pan India. Roles and Responsibility Design and develop scalable big data systems using Pyspark and Azure Data Bricks. Implement and manage workflows for efficient data processing. Collaborate with cross-functional teams to integrate data from various sources. Develop and maintain large-scale data pipelines and architectures. Optimize system performance and troubleshoot issues. Ensure data quality and integrity through data validation and testing procedures. Job Requirements S...

Job Requirements Role/ Job Title: Data Engineer 2 Function/ Department: Data & Analytics Job Purpose The Data Engineer is responsible for developing day to day data engineering projects and aiding in building the business’ data collection systems and processing pipelines. Responsible for efficient code for building and maintaining optimized and highly available data pipelines that facilitate deeper analysis and reporting by the Data and Analytics department. He/She works with Data and Analytics teams in leveraging data with reporting and scientific tools, for example, Tableau, Python, and Spark. The Data Engineer strives to continuously develop most efficient code for data pipeline or use ca...

Responsibilities Architect and design high-performance, scalable software solutions. Drive the end-to-end software development lifecycle from brainstorming to deployment. Build robust tools and applications by writing clean, efficient, and reusable code. Automate workflows using modern tools and scripting for seamless processes. Debug, review, and optimize code to ensure top-notch performance. Conduct validation and verification testing for stable, high-quality products. Collaborate with cross-functional teams and vendors to enhance products continuously. Document development phases and maintain system health. Stay ahead by adopting cutting-edge technologies and trends. Requirements Proven e...

Should be capable of developing/configuring data pipelines in a variety of platforms and technologies Possess the following technical skills SQL, Python, Pyspark, Hive, ETL, Unix, Control-M (or similar scheduling tools) Can demonstrate strong experience in writing complex SQL queries to perform data analysis on Databases SQL server, Oracle, HIVE etc. Experience with GCP (particularly Airflow, Dataproc, Big Query) is an advantage Have experience with creating solutions which power AI/ML models and generative AI Ability to work independently on specialized assignments within the context of project deliverables Take ownership of providing solutions and tools that iteratively increase engineerin...

We are looking for a skilled Hadoop Developer with 3 to 6 years of experience to join our team in Chennai. The ideal candidate will have expertise in Scala, SQL, Unix, and Hive & Spark. Roles and Responsibility Design, develop, and implement scalable data processing solutions using Hadoop technologies. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data pipelines using Scala and Spark. Troubleshoot and resolve complex technical issues related to Hadoop applications. Participate in code reviews and contribute to improving overall code quality. Stay up-to-date with the latest trends and technologies in Hadoop developmen...

Python, big data (Hive, spark or snowflake, Kafka), SQL/ SQLServer. Experience in Python for data processing.

Immediate Openings onPyspark Experience : 5+ Skill:-Pyspark Location :- Bangalore Notice Period :- Immediate. Pyspark Experience in Cloud platform, e.g., AWS, GCP, Azure, etc. Experience in distributed technology tools, viz. SQL, Spark, Python, PySpark, Scala Performance Turing Optimize SQL, PySpark for performance Airflow workflow scheduling tool for creating data pipelines GitHub source control tool & experience with creating/ configuring Jenkins pipeline

Role Description As a Technical Lead - Cloud Data Platform (GCP) at Incedo, you will be responsible for managing and optimizing the Google Cloud Platform environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or Azure and have experience with big data technologies such as Hadoop or Spark. You will be responsible for configuring and optimizing the GCP environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the security team...

C1X AdTech Pvt Ltd is a fast-growing, product and engineering-driven AdTech company building next-generation advertising and marketing technology platforms. Our mission is to empower enterprise clients with the smartest marketing solutions, enabling seamless integration with personalization engines and delivering cross-channel marketing capabilities. We are dedicated to enhancing customer engagement and experiences while focusing on increasing Lifetime Value (LTV) through consistent messaging across all channels. Our engineering team spans front end (UI), back end (Java/Node.js APIs), Big Data, and DevOps, working together to deliver scalable, high-performance products for the digital advert...

As a Data Engineer at Valorem Reply, you will be responsible for designing and contributing to the architecture of database systems, including both normalized transactional systems and dimensional reporting systems. Your expertise with SQL Server and Microsoft technologies, as well as experience in implementing Big Data solutions using Databricks, will be essential for success in this role. You will play a key role in providing data solutions to support business applications and reporting systems, working closely with customers to deliver effective solutions. Proficiency in Power BI and Azure Data Services is a must to excel in this position. Key Responsibilities: - Design and implement arch...

As a Java with Hadoop Developer at Airlinq, you will be part of a team responsible for building end-to-end IoT solutions using Hadoop to address various industry challenges. Your key responsibilities will include: - Building quick prototypes and demonstrations to showcase the value of technologies like IoT, Machine Learning, Cloud, Micro-Services, DevOps, and AI to the management team. - Developing reusable components, frameworks, and accelerators to streamline the development lifecycle of future IoT projects. - Working with minimal supervision to contribute effectively to the team's goals. - Setting up Cloud platforms for different use-cases. To be successful in this role, you must meet the...

As a Big Data Engineer, you will be responsible for working on collecting, storing, processing, and analyzing huge sets of data. Your primary focus will be on choosing optimal solutions for these purposes, then maintaining, implementing, and monitoring them. Additionally, you will integrate these solutions with the architecture used across the company. Key Responsibilities: - 6+ years" experience in Big Data Hadoop, MapReduce, Hive, Sqoop, and Spark with hands-on expertise in designing and implementing high data volume solutions (ETL & Streaming). - Experience in building stream-processing systems using solutions like Storm, Spark-Streaming, and Kafka streams. - Extensive experience working ...

Overview Join our team as an Associate Product Support Engineer focused exclusively on Hadoop distributed systems. In this customer-facing role, you will support enterprise clients by troubleshooting and resolving issues within Hadoop environments. You will work closely with experienced engineers, ensuring the stability and reliability of our customers’ Hadoop infrastructures. Key Responsibilities Provide technical support for Hadoop clusters, addressing customer issues with empathy and professionalism. Troubleshoot and resolve problems related to Hadoop core components (HDFS, YARN, MapReduce, Hive, etc.). Perform basic administrative tasks such as installation, configuration, and upgrades o...

Salary As per the industry standard Location Kolkata Job Information Description A Big Data Developer needs to have a knowledge of core Java, SQL, and any scripting language along with good interpersonal skills. Big data developer is responsible for the actual coding or programming of Hadoop, Hive, Kafka & InfluxDB Applications. Skills required to become a Big data developer: Knowledge related to Hadoop, Hive, Kafka & InfluxDB Good knowledge in back‐end programming, specifically Java, JS, Node.js and OOAD Good knowledge of database structures, theories, principles, and practices. Ability to write MapReduce jobs. Analytical and problem-solving skills High‐performance, reliable and maintainabl...

As a Staff Software Engineer at Visa, you will be a part of the Visa Risk Manager team focusing on creating new capabilities for the Risk Platform. Your main responsibilities will include troubleshooting, problem-solving, designing, building, and maintaining solutions for clients. You will need deep knowledge of computer science fundamentals, mobile & web technologies, multi-tiered architecture, and experience in tools development. In addition to technical skills, you should be a self-starter with excellent communication skills to collaborate with external teams and stakeholders. Mentoring junior software engineers and out-of-the-box thinking will be key aspects of your role. Key Responsibil...

Job Description The Lead Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment. Responsibilities: Responsible for design and development of big data solutions. Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in Hadoop or Snowflake Responsible for delivering data as a service framework Responsible for moving all legacy workloads to cloud platform Work with data scientist to build Client pipelines using heterogeneous sour...

Looking to onboard a skilled Hadoop Admin with 5-12 years of experience to join our team in Bangalore. The ideal candidate will have a strong background in Big Data technologies and programming languages such as Python, Scala. Roles and Responsibility Design, develop, and implement scalable data processing systems using Hadoop and Spark. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data pipelines and architectures. Troubleshoot and resolve complex technical issues related to data processing and storage. Ensure high availability and performance of data systems and applications. Participate in code reviews and contrib...

Chennai / HYD / Kolkata / Delhi / Pune / BLR Functional Skills: Experience in Credit Risk/Regulatory risk domain Technical Skills: Spark ,PySpark, Python, Hive, Scala, MapReduce, Unix shell scripting Machine learning tech.

What youll do: We seek Software Engineers with experience building and scaling services in on-premises and cloud environments. As a Principal Software Engineer in the Epsilon Attribution/Forecasting Product Development team, you will design, implement, and optimize data processing solutions using Scala, Spark, and Hadoop. Collaborate with cross-functional teams to deploy big data solutions on our on-premises and cloud infrastructure along with building, scheduling and maintaining workflows. Perform data integration and transformation, troubleshoot issues, Document processes, communicate technical concepts clearly, and continuously enhance our attribution engine/forecasting engine. Strong wri...

Design and implement an end-to-end data solution architecture tailored to enterprise analytics and operational needs. Build, maintain, and optimize data pipelines and transformations using Python, SQL, and Spark . Manage large-scale data storage and processing with Iceberg, Hadoop, and HDFS . Develop and maintain dbt models to ensure clean, reliable, and well-structured data. Implement robust data ingestion processes , integrating with third-party APIs and on-premise systems. Collaborate with cross-functional teams to align on business goals and technical requirements. Contribute to documentation and continuously improve engineering and data quality processes. Qualifications 4+ years of expe...

JD:- Total Yrs. of Experience* 6+ Relevant Yrs. of experience* 6+ Detailed JD *(Roles and Responsibilities) 6+ years experience in Kafka, Spark, Scala ,Pyspark Strong knowledge of implementing and maintaining scalable streaming and batch data solutions with high throughput and strict SLA. Extensive experience of working in production with Kafka, Spark and Kafka Stream including best practices, observability, optimization, performance tuning. Knowledge of Python and Scala including best practices, architecture, dependency management Collaborate with cross-functional teams to understand data requirements and integrate data from multiple sources, ensuring data consistency and quality Mandatory ...

JD:- Total Yrs. of Experience* 6+ Relevant Yrs. of experience* 6+ Detailed JD *(Roles and Responsibilities) 6+ years experience in Kafka, Spark, Scala ,Pyspark Strong knowledge of implementing and maintaining scalable streaming and batch data solutions with high throughput and strict SLA. Extensive experience of working in production with Kafka, Spark and Kafka Stream including best practices, observability, optimization, performance tuning. Knowledge of Python and Scala including best practices, architecture, dependency management Collaborate with cross-functional teams to understand data requirements and integrate data from multiple sources, ensuring data consistency and quality Mandatory ...

Skill:Data Engineer Notice Period:Immediate. Employment Type: Contract Job Description : Experience in Cloud platform, e.g., AWS, GCP, Azure, etc. Experience in distributed technology tools, viz. SQL, Spark, Python, PySpark, Scala Performance Turing Optimize SQL, PySpark for performance Airflow workflow scheduling tool for creating data pipelines GitHub source control tool & experience with creating/ configuring Jenkins pipeline Experience in EMR/ EC2, Databricks etc. DWH tools incl. SQL database, Presto, and Snowflake Streaming, Serverless Architecture .

Notice Period: Immediate Joiners Job Description: Must have skills Azure DataBricks, Python And Pyspark, Spark. Please find the JD In the mail chain Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on experience in Azure Databricks , Data Factory, Data Lake store/Blob storage, SQL DB Experience in creating Big data Pipelines with Azure components Hands on programing with Python Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming Experience with messaging systems, such as Kafka or RabbitMQ Good understanding of...

Big data engineer/Developer Spark-Scala HQL, Hive Control-m Jinkins Git Technical analysis and up to some extent business analysis (knowledge about banking products, credit cards and its transactions) Notice Period: Immediate . Employment Type: Contract