Home
Jobs

4958 Hadoop Jobs - Page 44

Filter
Filter Interviews
Min: 0 years
Max: 25 years
Min: ₹0
Max: ₹10000000
Setup a job Alert
JobPe aggregates results for easy application access, but you actually apply on the job portal directly.

2.0 years

0 Lacs

Bengaluru East, Karnataka, India

On-site

Linkedin logo

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid. Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa. Job Description Functional Summary The GTM Optimization and Business Health team has a simple mission: we turn massive amounts of data into robust tools and actionable insights that drive business value, ensure ecosystem integrity, and provide best in class experience to our money movement clients. Our team is working to build consolidated, strategic and scalable analytics and monitoring infrastructure for commercial and money movement products. Responsibilities The Process Optimization Analyst will create risk, rules, and performance monitoring dashboards and alerting tools and will use these to monitor transactions in near real time, investigate alerts and anomalous events, and partner with internal teams to investigate and manage incidents from end-to-end. Specific activities may include: Develop monitoring and alerting tools from real-time data feeds to monitor for performance drops, risk and fraud events, and rules violations Monitor near real time alerting tools and investigate and generate incidents for risk events and out of pattern activity Manage a caseload to ensure appropriate investigation and resolution of identified risk and performance events Drive to understand the root problems, define analytical objectives and formalize data requirements for various types of dashboards and analyses Design and launch robust and intuitive dashboards supporting best in class money movement client experience Create and present analytic deliverables to colleagues in the analytics team, other internal stakeholders with varying degrees of analytical and technical expertise Distill massive amounts of data across disparate data sources into efficient functional data repositories in a Big Data environment Independently perform analysis to derive insights and render robust, thoughtful results Partner with Visa Direct and money movement teams across multiple areas of the business to understand their data and reporting needs Compare client performance against industry best practices with a shrewd eye toward identifying performance and/or profitability improvement opportunity Develop presentations of complex data and content for clients in an accurate, understandable, and engaging manner This is a hybrid position. Expectation of days in office will be confirmed by your Hiring Manager. Qualifications Basic Qualifications: • 3 or more years of relevant work experience with a Bachelor’s Degree or at least 2 years of work experience with an Advanced degree (e.g. Masters, MBA, JD, MD) or 0 years of work experience with a PhD Preferred Qualifications: • 3 or more years of work experience with a Bachelor’s Degree or 2 or more years of relevant experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or up to 1 years of relevant experience with a PhD • Experience monitoring real-time data and following incident management workflows • Familiarity with Microsoft Dynamics or other ERP/CRM tools • Proficiency in Tableau and experience with best in class data visualization • Experience with Elasticsearch and Kibana dashboard and alerting • High level of proficiency manipulating data from a variety of sources - Big data skills (Hadoop, Hive, Spark) and/or SQL skills required • Strong verbal, written, and interpersonal skills • Proficient in all MS Office applications with advanced Excel spreadsheet skills • Functional knowledge of programming languages such as Python, Java, and/or Shell Scripting • Strong strategic thinking, problem-solving, and decision-making abilities, with the ability to translate complex data into actionable insights • Visa experience or knowledge of the payments industry Additional Information Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law. Show more Show less

Posted 1 week ago

Apply

5.0 - 8.0 years

0 Lacs

Bengaluru, Karnataka, India

On-site

Linkedin logo

About the role The Data Analyst in the GRP team will be responsible to analyse complex datasets and make it consumable using visual storytelling and visualization tools such as reports and dashboards built using approved tools (Tableau, Microstrategy, PyDash). The ideal candidate will have a strong analytical mindset, excellent communication skills, and a deep understanding of reporting tools front end and back end You will be responsible for - Driving Data analysis for testing key business hypothesis and asks, developing complex visualizations, self-service tools and cockpits for answering recurring business asks and measurements - Experience in handling quick turnaround business requests, managing stakeholder communication and solving business asks holistically going beyond the basic stakeholder asks - Ability to select the right tools and techniques for solving the problem in hand - Ensuring analysis, tools/ dashboards are developed with the right technical rigor meeting Tesco technical standards - Applied experience in handling large data-systems and datasets - Extensive experience in handling high volume, time pressured business asks and ad-hocs requests - Ability to develop production ready visualization solutions and automated reports - Contribute to development of knowledge assets and reusable modules on GitHub/Wiki- Come up with new ideas and analysis to support business priorities and solve business problems You will need 5-8 years of experience as a Data Analyst, with experience working in domains like retail, cpg and for one of the following functional areas – Finacne, marketing, supply chain, customer, merchandising preferred - Proven track record of handling ad-hoc analysis, developing dashboards and visualizations based business asks. - Strong usage of business understanding for analysis asks. - Exposure to analysis work within Retail domain; Space, Range, Merchandising, Store Ops, Forecasting, Customer Insights, Digital, Marketing will be preferred- Expert Skills to analyze large datasets using Adv Excel, Adv SQL, Hive, Phython, - Expert Skills to develop visualizations, self-service dashboards and reports using Tableau & PowerBi, - Statistical Concepts (Correlation Analysis and Hyp. Testing), Strong DW concepts (Hadoop, Teradata), - Excellent analytical and problem-solving skills. - Should be comfortable dealing with variability- Strong communication and interpersonal skills. Whats in it for you? At Tesco, we are committed to providing the best for you. As a result, our colleagues enjoy a unique, differentiated, market- competitive reward package, based on the current industry practices, for all the work they put into serving our customers, communities and planet a little better every day. Our Tesco Rewards framework consists of pillars - Fixed Pay, Incentives, and Benefits. Total Rewards offered at Tesco is determined by four principles - simple, fair, competitive, and sustainable. Salary - Your fixed pay is the guaranteed pay as per your contract of employment. Performance Bonus - Opportunity to earn additional compensation bonus based on performance, paid annually Leave & Time-off - Colleagues are entitled to 30 days of leave (18 days of Earned Leave, 12 days of Casual/Sick Leave) and 10 national and festival holidays, as per the company’s policy. Making Retirement Tension-FreeSalary - In addition to Statutory retirement beneets, Tesco enables colleagues to participate in voluntary programmes like NPS and VPF. Health is Wealth - Tesco promotes programmes that support a culture of health and wellness including insurance for colleagues and their family. Our medical insurance provides coverage for dependents including parents or in-laws. Mental Wellbeing - We offer mental health support through self-help tools, community groups, ally networks, face-to-face counselling, and more for both colleagues and dependents. Financial Wellbeing - Through our financial literacy partner, we offer one-to-one financial coaching at discounted rates, as well as salary advances on earned wages upon request. Save As You Earn (SAYE) - Our SAYE programme allows colleagues to transition from being employees to Tesco shareholders through a structured 3-year savings plan. Physical Wellbeing - Our green campus promotes physical wellbeing with facilities that include a cricket pitch, football field, badminton and volleyball courts, along with indoor games, encouraging a healthier lifestyle. About Us Tesco in Bengaluru is a multi-disciplinary team serving our customers, communities, and planet a little better every day across markets. Our goal is to create a sustainable competitive advantage for Tesco by standardising processes, delivering cost savings, enabling agility through technological solutions, and empowering our colleagues to do even more for our customers. With cross-functional expertise, a wide network of teams, and strong governance, we reduce complexity, thereby offering high-quality services for our customers. Tesco in Bengaluru, established in 2004 to enable standardisation and build centralised capabilities and competencies, makes the experience better for our millions of customers worldwide and simpler for over 3,30,000 colleagues. Tesco Business Solutions: Established in 2017, Tesco Business Solutions (TBS) has evolved from a single entity traditional shared services in Bengaluru, India (from 2004) to a global, purpose-driven solutions-focused organisation. TBS is committed to driving scale at speed and delivering value to the Tesco Group through the power of decision science. With over 4,400 highly skilled colleagues globally, TBS supports markets and business units across four locations in the UK, India, Hungary, and the Republic of Ireland. The organisation underpins everything that the Tesco Group does, bringing innovation, a solutions mindset, and agility to its operations and support functions, building winning partnerships across the business. TBS's focus is on adding value and creating impactful outcomes that shape the future of the business. TBS creates a sustainable competitive advantage for the Tesco Group by becoming the partner of choice for talent, transformation, and value creation Show more Show less

Posted 1 week ago

Apply

3.0 years

0 Lacs

Bengaluru, Karnataka, India

On-site

Linkedin logo

It's fun to work in a company where people truly BELIEVE in what they are doing! We're committed to bringing passion and customer focus to the business. Job Description Data Engineer (AWS) Role : DE Engineer (AWS) Experience : 3-5 years (3+ years of experience with AWS cloud) Education : BE/B. Tech/M. Tech Location : Bangalore/India We are currently seeking an experienced Data Support Engineer with a focus on AWS, Snowflake, Hadoop, Spark, and Python to join our Support team. The ideal candidate will have a solid technical background, strong problem-solving skills, and hands-on experience in troubleshooting and supporting data engineering systems. Responsibilities Include Hands-on experience with Hadoop, Spark with Python on AWS. Provide technical support for data engineering systems, addressing user queries, and resolving issues related to data pipelines, AWS services, Snowflake, Hadoop, and Spark. Investigate and troubleshoot issues in data pipelines, identifying root causes and implementing solutions to prevent recurrence. Experience with a range of big data architectures like Hadoop,Spark,Kafka,Hive or other big data technologies. Effectively manage and resolve incidents related to data processing, ensuring minimal downtime and optimal system performance. Collaborate with cross-functional teams to prioritize and address critical issues promptly. Experience in Tuning and Optimizing Spark jobs Knowledge on Terraform templates for Infrastructure provisioning on AWS (or cloud formation templates) Possess minimum 3+ years of in BI/DW development experience with Data Model Architecture/Design. Should have a good understanding of functional programming concepts. Good knowledge of Python with experience of production grade Python projects. Continuous Integration, branching and merging, pair programming, code reviews, unit testing, agile methodologies (Scrum), Design Patterns. Knowledge on CI/CD implementation like AWS Code Commit, Code Deploy for CI/CD pipelines (Git knowledge preferable) Knowledge on Scheduling Tools and techniques on Hadoop/EMR. Excellent written and verbal communication skills. Strong analytical and project management skills. Technical Essentials Proven experience in providing technical support for data engineering systems. Strong understanding of AWS services, including S3, Glue, Redshift, EMR, Lambda, Athena, and Step Functions. Hands-on experience supporting Snowflake, Hadoop, Spark, and Python in a production environment. Familiarity with data modeling, optimization, and performance tuning. Excellent problem-solving skills and the ability to analyze and diagnose complex technical issues. Experience with incident management, including prioritization and resolution procedures. Strong communication and collaboration skills for working with cross-functional teams. Knowledge of best practices in cloud-based data engineering and support. Preferred AWS Certified Solutions Architect – Associate Personal Specifications Self-motivated team player with strong analytical, relationship management with effective written and oral communication skills. If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us! Not the right fit? Let us know you're interested in a future opportunity by clicking Introduce Yourself in the top-right corner of the page or create an account to set up email alerts as new job postings become available that meet your interest! Show more Show less

Posted 1 week ago

Apply

8.0 - 10.0 years

0 Lacs

Pune, Maharashtra, India

On-site

Linkedin logo

Job description: Job Description Role Purpose The purpose of the role is to create exceptional architectural solution design and thought leadership and enable delivery teams to provide exceptional client engagement and satisfaction. ͏ Do 1.Develop architectural solutions for the new deals/ major change requests in existing deals Creates an enterprise-wide architecture that ensures systems are scalable, reliable, and manageable. Provide solutioning of RFP’s received from clients and ensure overall design assurance Develop a direction to manage the portfolio of to-be-solutions including systems, shared infrastructure services, applications in order to better match business outcome objectives Analyse technology environment, enterprise specifics, client requirements to set a collaboration solution design framework/ architecture Provide technical leadership to the design, development and implementation of custom solutions through thoughtful use of modern technology Define and understand current state solutions and identify improvements, options & tradeoffs to define target state solutions Clearly articulate, document and sell architectural targets, recommendations and reusable patterns and accordingly propose investment roadmaps Evaluate and recommend solutions to integrate with overall technology ecosystem Works closely with various IT groups to transition tasks, ensure performance and manage issues through to resolution Perform detailed documentation (App view, multiple sections & views) of the architectural design and solution mentioning all the artefacts in detail Validate the solution/ prototype from technology, cost structure and customer differentiation point of view Identify problem areas and perform root cause analysis of architectural design and solutions and provide relevant solutions to the problem Collaborating with sales, program/project, consulting teams to reconcile solutions to architecture Tracks industry and application trends and relates these to planning current and future IT needs ͏ Provides technical and strategic input during the project planning phase in the form of technical architectural designs and recommendation Collaborates with all relevant parties in order to review the objectives and constraints of solutions and determine conformance with the Enterprise Architecture Identifies implementation risks and potential impacts 2.Enable Delivery Teams by providing optimal delivery solutions/ frameworks Build and maintain relationships with executives, technical leaders, product owners, peer architects and other stakeholders to become a trusted advisor Develops and establishes relevant technical, business process and overall support metrics (KPI/SLA) to drive results Manages multiple projects and accurately reports the status of all major assignments while adhering to all project management standards Identify technical, process, structural risks and prepare a risk mitigation plan for all the projects Ensure quality assurance of all the architecture or design decisions and provides technical mitigation support to the delivery teams Recommend tools for reuse, automation for improved productivity and reduced cycle times Leads the development and maintenance of enterprise framework and related artefacts Develops trust and builds effective working relationships through respectful, collaborative engagement across individual product teams Ensures architecture principles and standards are consistently applied to all the projects Ensure optimal Client Engagement Support pre-sales team while presenting the entire solution design and its principles to the client Negotiate, manage and coordinate with the client teams to ensure all requirements are met and create an impact of solution proposed Demonstrate thought leadership with strong technical capability in front of the client to win the confidence and act as a trusted advisor ͏ 3.Competency Building and Branding Ensure completion of necessary trainings and certifications Develop Proof of Concepts (POCs),case studies, demos etc. for new growth areas based on market and customer research Develop and present a point of view of Wipro on solution design and architect by writing white papers, blogs etc. Attain market referencability and recognition through highest analyst rankings, client testimonials and partner credits Be the voice of Wipro’s Thought Leadership by speaking in forums (internal and external) Mentor developers, designers and Junior architects in the project for their further career development and enhancement Contribute to the architecture practice by conducting selection interviews etc ͏ 4.Team Management Resourcing Anticipating new talent requirements as per the market/ industry trends or client requirements Hire adequate and right resources for the team Talent Management Ensure adequate onboarding and training for the team members to enhance capability & effectiveness Build an internal talent pool and ensure their career progression within the organization Manage team attrition Drive diversity in leadership positions Performance Management Set goals for the team, conduct timely performance reviews and provide constructive feedback to own direct reports Ensure that the Performance Nxt is followed for the entire team Employee Satisfaction and Engagement Lead and drive engagement initiatives for the team Track team satisfaction scores and identify initiatives to build engagement within the team Mandatory Skills: Hadoop . Experience: 8-10 Years . Reinvent your world. We are building a modern Wipro. We are an end-to-end digital transformation partner with the boldest ambitions. To realize them, we need people inspired by reinvention. Of yourself, your career, and your skills. We want to see the constant evolution of our business and our industry. It has always been in our DNA - as the world around us changes, so do we. Join a business powered by purpose and a place that empowers you to design your own reinvention. Come to Wipro. Realize your ambitions. Applications from people with disabilities are explicitly welcome. Show more Show less

Posted 1 week ago

Apply

5.0 - 10.0 years

7 - 12 Lacs

Pune

Work from Office

Naukri logo

The data architect is responsible for designing, creating, and managing an organizations data architecture. This role is critical in establishing a solid foundation for data management within an organization, ensuring that data is organized, accessible, secure, and aligned with business objectives. The data architect designs data models, warehouses, file systems and databases, and defines how data will be collected and organized. Responsibilities Interprets and delivers impactful strategic plans improving data integration, data quality, and data delivery in support of business initiatives and roadmaps Designs the structure and layout of data systems, including databases, warehouses, and lakes Selects and designs database management systems that meet the organizations needs by defining data schemas, optimizing data storage, and establishing data access controls and security measures Defines and implements the long-term technology strategy and innovations roadmaps across analytics, data engineering, and data platforms Designs processes for the ETL process from various sources into the organizations data systems Translates high-level business requirements into data models and appropriate metadata, test data, and data quality standards Manages senior business stakeholders to secure strong engagement and ensures that the delivery of the project aligns with longer-term strategic roadmaps Simplifies the existing data architecture, delivering reusable services and cost-saving opportunities in line with the policies and standards of the company Leads and participates in the peer review and quality assurance of project architectural artifacts across the EA group through governance forums Defines and manages standards, guidelines, and processes to ensure data quality Works with IT teams, business analysts, and data analytics teams to understand data consumers needs and develop solutions Evaluates and recommends emerging technologies for data management, storage, and analytics Design, create, and implement logical and physical data models for both IT and business solutions to capture the structure, relationships, and constraints of relevant datasets Build and operationalize complex data solutions, correct problems, apply transformations, and recommend data cleansing/quality solutions Effectively collaborate and communicate with various stakeholders to understand data and business requirements and translate them into data models Create entity-relationship diagrams (ERDs), data flow diagrams, and other visualization tools to represent data models Collaborate with database administrators and software engineers to implement and maintain data models in databases, data warehouses, and data lakes Develop data modeling best practices, and use these standards to identify and resolve data modeling issues and conflicts Conduct performance tuning and optimization of data models for efficient data access and retrieval Incorporate core data management competencies, including data governance, data security and data quality Job Requirements Education: A bachelors degree in computer science, data science, engineering, or related field Experience: At least five years of relevant experience in design and implementation of data models for enterprise data warehouse initiatives Experience leading projects involving data warehousing, data modeling, and data analysis Design experience in Azure Databricks, PySpark, and Power BI/Tableau Skills: Ability in programming languages such as Java, Python, and C/C++ Ability in data science languages/tools such as SQL, R, SAS, or Excel Proficiency in the design and implementation of modern data architectures and concepts such as cloud services (AWS, Azure, GCP), real-time data distribution (Kafka, Dataflow), and modern data warehouse tools (Snowflake, Databricks) Experience with database technologies such as SQL, NoSQL, Oracle, Hadoop, or Teradata Understanding of entity-relationship modeling, metadata systems, and data quality tools and techniques Ability to think strategically and relate architectural decisions and recommendations to business needs and client culture Ability to assess traditional and modern data architecture components based on business needs Experience with business intelligence tools and technologies such as ETL, Power BI, and Tableau Ability to regularly learn and adopt new technology, especially in the ML/AI realm Strong analytical and problem-solving skills Ability to synthesize and clearly communicate large volumes of complex information to senior management of various technical understandings Ability to collaborate and excel in complex, cross-functional teams involving data scientists, business analysts, and stakeholders Ability to guide solution design and architecture to meet business needs Expert knowledge of data modeling concepts, methodologies, and best practices Proficiency in data modeling tools such as Erwin or ER/Studio Knowledge of relational databases and database design principles Familiarity with dimensional modeling and data warehousing concepts Strong SQL skills for data querying, manipulation, and optimization, and knowledge of other data science languages, including JavaScript, Python, and R Ability to collaborate with cross-functional teams and stakeholders to gather requirements and align on data models Excellent analytical and problem-solving skills to identify and resolve data modeling issues Strong communication and documentation skills to effectively convey complex data modeling concepts to technical and business stakeholders

Posted 1 week ago

Apply

0.0 - 3.0 years

0 Lacs

Hyderabad, Telangana

On-site

Indeed logo

Naresh IT KPHB & Ameerpet branches are hiring for an experienced Data Science trainers. Candidates Should have a good knowledge of Machine learning , Deep Learning , R Programming; Python, Statistics, Hadoop etc. Mandatory to have training experience Job Type: Full-time Pay: ₹500,000.00 - ₹800,000.00 per year Experience: Software Training: 3 years (Preferred) Location: Hyderabad, Telangana (Preferred) Work Location: In person

Posted 1 week ago

Apply

3.0 - 8.0 years

7 - 17 Lacs

Mumbai, Pune

Hybrid

Naukri logo

Role: Senior Data Engineer Location: Mumbai & Pune Experience: 3yrs to 8yrs Technologies / Skills: Advanced SQL, Python and associated libraries like Pandas, NumPy etc., Pyspark , Shell scripting, Data-Modelling, Big data, Hadoop, Hive, ETL pipelines. Responsibilities: • Proven success in communicating with users, other technical teams, and senior management to collect requirements, describe data modeling decisions and develop data engineering strategy. • Ability to work with business owners to define key business requirements and convert to user stories with required technical specifications. • Communicate results and business impacts of insight initiatives to key stakeholders to collaboratively solve business problems. • Working closely with the overall Enterprise Data & Analytics Architect and Engineering practice leads to ensure adherence with the best practices and design principles. • Assures quality, security and compliance requirements are met for supported area. • Design and create fault-tolerance data pipelines running on cluster • Excellent communication skills with the ability to influence client business and IT teams • Should have design data engineering solutions end to end. Ability to come up with scalable and modular solutions. Required Qualification: • 3+ years of hands-on experience Designing and developing Data Pipelines for Data Ingestion or Transformation using Python (PySpark)/Spark SQL in AWS cloud • Experience in design and development of data pipelines and processing of data at scale. • Advanced experience in writing and optimizing efficient SQL queries with Python and Hive handling Large Data Sets in Big-Data Environments • Experience in debugging, tunning and optimizing PySpark data pipelines • Should have implemented concepts and have good knowledge of Pyspark data frames, joins, caching, memory management, partitioning, parallelism etc. • Understanding of Spark UI, Event Timelines, DAG, Spark config parameters, in order to tune the long running data pipelines. • Experience working in Agile implementations • Experience with building data pipelines in streaming and batch mode. • Experience with Git and CI/CD pipelines to deploy cloud applications • Good knowledge of designing Hive tables with partitioning for performance. Desired Qualification: • Experience in data modelling. • Hands on creating workflows on any Scheduling Tool like Autosys, CA Workload Automation. • Proficiency in using SDKsfor interacting with native AWS services. • Strong understanding of concepts of ETL, ELT and data modeling.

Posted 1 week ago

Apply

3.0 years

0 Lacs

Bengaluru, Karnataka, India

On-site

Linkedin logo

Key Responsibilities Partner with product managers, engineers, and business stakeholders to define KPIs and success metrics for Creator Success Create comprehensive dashboards and self-service analytics tools using QuickSight, Tableau, or similar BI platforms Perform deep-dive analysis on customer behavior, content performance, and livestream engagement patterns Design, build, and maintain robust ETL/ELT pipelines to process large volumes of streaming and batch data from Creator Success platform Develop and optimize data warehouses, data lakes, and real-time analytics systems using AWS services (Redshift, S3, Kinesis, EMR, Glue) Implement data quality frameworks and monitoring systems to ensure data accuracy and reliability Build automated data validation and alerting mechanisms for critical business metrics Generate actionable insights from complex datasets to drive product roadmap and business strategy Required Qualifications Bachelor's degree in Computer Science, Engineering, Mathematics, Statistics, or related quantitative field 3+ years of experience in business intelligence/analytic roles with proficiency in SQL, Python, and/or Scala Strong experience with AWS cloud services (Redshift, S3, EMR, Glue, Lambda, Kinesis) Expertise in building and optimizing ETL pipelines and data warehousing solutions Proficiency with big data technologies (Spark, Hadoop) and distributed computing frameworks Experience with business intelligence tools (QuickSight, Tableau, Looker) and data visualization best practices Collaborative approach with cross-functional teams including product, engineering, and business teams Customer-obsessed mindset with focus on delivering high-quality, actionable insights Non-Negotiable Skills High proficiency in SQL and Python Expertise in building and optimizing ETL pipelines and data warehousing solutions Experience with business intelligence tools (QuickSight, Tableau, Looker) and data visualization best practices Experience in working with cross-functional teams including product, engineering, and business teams Experience with AWS cloud services (Redshift, S3, EMR) Show more Show less

Posted 1 week ago

Apply

8.0 - 12.0 years

8 - 13 Lacs

Bengaluru

Work from Office

Naukri logo

Happiest Minds Technologies Pvt.Ltd is looking for Sr Data and ML Engineer to join our dynamic team and embark on a rewarding career journey Liaising with coworkers and clients to elucidate the requirements for each task. Conceptualizing and generating infrastructure that allows big data to be accessed and analyzed. Reformulating existing frameworks to optimize their functioning. Testing such structures to ensure that they are fit for use. Preparing raw data for manipulation by data scientists. Detecting and correcting errors in your work. Ensuring that your work remains backed up and readily accessible to relevant coworkers. Remaining up-to-date with industry standards and technological advancements that will improve the quality of your outputs. Spark ML Lib,Scala,Python,Databricks on AWS, Snowflake, GitLab, Jenkins, AWS DevOps CI/CD pipeline, Machine Learning, Airflow

Posted 1 week ago

Apply

3.0 - 6.0 years

6 - 10 Lacs

Bengaluru

Work from Office

Naukri logo

Data Engineering Pipeline Development Design implement and maintain ETL processes using ADF and ADB Create and manage views in ADB and SQL for efficient data access Optimize SQL queries for large datasets and high performance Conduct end-to-end testing and impact analysis on data pipelines Optimization Performance Tuning Identify and resolve bottlenecks in data processing Optimize SQL queries and Delta Tables for fast data processing Data Sharing Integration Implement Delta Share, SQL Endpoints, and other data sharing methods Use Delta Tables for efficient data sharing and processing API Integration Development Integrate external systems through Databricks Notebooks and build scalable solutions Experience in building APIs (Good to have) Collaboration Documentation Collaborate with teams to understand requirements and design solutions Provide documentation for data processes and architectures

Posted 1 week ago

Apply

2.0 - 9.0 years

10 - 14 Lacs

Bengaluru

Work from Office

Naukri logo

Design, develop, and maintain scalable and efficient Python applications using frameworks like FastAPI or Flask Develop, test, and deploy RESTful APIs to interact with front-end services Integrate and establish connections between various relational and non-relational databases (eg, SQL Alchemy, MySQL, PostgreSQL, MongoDB, etc) Solid understanding of relational and NoSQL databases and the ability to establish and manage connections from Python applications Write clean, maintainable, and efficient code, following coding standards and best practices Leverage AWS cloud services for deploying and managing applications (eg, EC2, Lambda, RDS, S3, etc) Troubleshoot and resolve software defects, performance issues, and scalability challenges

Posted 1 week ago

Apply

5.0 - 8.0 years

10 - 14 Lacs

Bengaluru

Work from Office

Naukri logo

BS or higher degree in Computer Science (or equivalent field) 3-6+ years of programming experience with Java and Python Strong in writing SQL queries and understanding of Kafka, Scala, Spark/Flink Exposure to AWS Lambda, AWS Cloud Watch, Step Functions, EC2, Cloud Formation, Jenkins

Posted 1 week ago

Apply

4.0 - 7.0 years

5 - 9 Lacs

Bengaluru

Work from Office

Naukri logo

PySpark Python SQL Strong focus on big data processing which is core to data engineering AWS Cloud Services Lambda Glue S3 IAMIndicates working with cloud based data pipelines Airflow GitHub Essential for orchestration and version control in data workflows

Posted 1 week ago

Apply

3.0 - 7.0 years

6 - 10 Lacs

Bengaluru

Work from Office

Naukri logo

Skills required : Bigdata Workflows (ETL/ELT), Python hands-on, SQL hands-on, Any Cloud (GCP BigQuery preferred), Airflow (good knowledge on Airflow features, operators, scheduling etc) NOTE Candidate will be having the coding test (Python and SQL) in the interview process. This would be done through coders-pad. Panel would set it at run-time.

Posted 1 week ago

Apply

10.0 - 15.0 years

8 - 13 Lacs

Bengaluru

Work from Office

Naukri logo

Good knowledge on Broadcast eco-system and content processing elements including workflows Hands-on in BMS, Traffic and Playout One globally renowed OEM product in each area Good Knowledge on dealing with currency data, Reports from Neilsen/BARC Good understanding of Sales function in broadcast including Traffic and currency, Affiliate, Non-Linear distribution Has worked/certified on cloud with experience on running and porting media systems into cloud. Knowledge on OEM products dealing with DAM/MAM/CMS Should have good understanding of content processing flow including pre-prod, prod and distribution. Good exposure to emerging technologies like Data Analytics and Gen AI in solving practical industry problems. Experience on content processing elements, streaming standards and protocols is an advantage. JD For Media Consultant Engages with customer and brings in value through prolific Solutioning. Be the domain consultant and act as bridge between customer and the delivery teams. Translate business requirements into clear and concise functional specifications and solutions for technical teams. Propose innovative and practical solutions to address market and business challenges. Work and develop relationships with partners, working with them to create market-led solutions. Constantly be on the lookout for ways to create solutions that deliver better value to the customers. Work with BDM and plan sales strategies in response to market and key accounts. Take ownership of opportunities and preparation of response to RFP/RFI or ad-hoc requirements working with other stake holders.

Posted 1 week ago

Apply

2.0 - 5.0 years

3 - 7 Lacs

Bengaluru

Work from Office

Naukri logo

Experience in designing and developing data pipelines in a modern data stack (Snowflake, AWS, Airflow,DBT etc) Strong experience on Python Over 2+ years of experience in Snowflake and DBT Able to work in afternoon shift and front end the customer independently so that he/she possess strong communication Strong knowledge in Python, DBT, Snowflake, Airflow Ability to manage both structured and unstructured data Work with multiple data sources (APIs, Databases, S3, et) Own design, documentation, and lifecycle management of data pipelines Help implement the CI/CD processes and release engineering for organizations data pipelines Experience in designing and developing CI/CD processes and managing release management for data pipelines Proficient in Python, SQL, Airflow, AWS, Bitbucket, working with APIs and other types of data sources Good to have knowledge in Salesforce Primary skills : AWS Cloud, Snowflake DW, Azure SQL, SQL, Python (Must Have) DBT( Must Have)

Posted 1 week ago

Apply

4.0 - 7.0 years

5 - 9 Lacs

Bengaluru

Work from Office

Naukri logo

PySpark, Python, SQL Strong focus on big data processing,which is core to data engineering. AWS Cloud Services (Lambda, Glue, S3, IAM) Indicates working with cloud-based data pipelines. Airflow, GitHub Essential for orchestration and version control in data workflows.

Posted 1 week ago

Apply

2.0 years

0 Lacs

Trivandrum, Kerala, India

On-site

Linkedin logo

Equifax is where you can power your possible. If you want to achieve your true potential, chart new paths, develop new skills, collaborate with bright minds, and make a meaningful impact, we want to hear from you. What you’ll do Perform general application development activities, including unit testing, code deployment to development environment and technical documentation. Works on one or more projects, making contributions to unfamiliar code written by team members. Participates in estimation process, use case specifications, reviews of test plans and test cases, requirements, and project planning. Diagnose and resolve performance issues. Documents code/processes so that any other developer is able to dive in with minimal effort. Develop, and operate high scale applications from the backend to UI layer, focusing on operational excellence, security and scalability. Apply modern software development practices (serverless computing, microservices architecture, CI/CD, infrastructure-as-code, etc.). Work across teams to integrate our systems with existing internal systems, Data Fabric, CSA Toolset. Participate in technology roadmap and architecture discussions to turn business requirements and vision into reality. Participate in a tight-knit engineering team employing agile software development practices. Triage product or system issues and debug/track/resolve by analyzing the sources of issues and the impact on network, or service operations and quality. Able to write, debug, and troubleshoot code in mainstream open source technologies. Lead effort for Sprint deliverables, and solve problems with medium complexity What Experience You Need Bachelor's degree or equivalent experience 2+ years experience working with software design and Java, SQL and Javascript programming languages 2+ years experience with software testing, performance, and quality engineering techniques and strategies 2+ years experience with Cloud technology: GCP, AWS, or Azure What Could Set You Apart Knowledge or experience with Apache Beam for stream and batch data processing. Familiarity with big data tools and technologies like Apache Kafka, Hadoop, or Spark. Experience with containerization and orchestration tools (e.g., Docker, Kubernetes). Exposure to data visualization tools or platforms. We offer a hybrid work setting, comprehensive compensation and healthcare packages, attractive paid time off, and organizational growth potential through our online learning platform with guided career tracks. Are you ready to power your possible? Apply today, and get started on a path toward an exciting new career at Equifax, where you can make a difference! Who is Equifax? At Equifax, we believe knowledge drives progress. As a global data, analytics and technology company, we play an essential role in the global economy by helping employers, employees, financial institutions and government agencies make critical decisions with greater confidence. We work to help create seamless and positive experiences during life’s pivotal moments: applying for jobs or a mortgage, financing an education or buying a car. Our impact is real and to accomplish our goals we focus on nurturing our people for career advancement and their learning and development, supporting our next generation of leaders, maintaining an inclusive and diverse work environment, and regularly engaging and recognizing our employees. Regardless of location or role, the individual and collective work of our employees makes a difference and we are looking for talented team players to join us as we help people live their financial best. Equifax is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran. Show more Show less

Posted 1 week ago

Apply

4.0 - 8.0 years

9 - 12 Lacs

Chennai

Work from Office

Naukri logo

Job Title: Data Engineer Location: Chennai (Hybrid) Summary Design,develop, and maintain scalable data pipelines and systems to support thecollection, integration, and analysis of healthcare and enterprise data. Theprimary responsibilities of this role include designing and implementingefficient data pipelines, architecting robust data models, and adhering to datamanagement best practices. In this position, you will play a crucial part intransforming raw data into meaningful insights, through development of semanticdata layers, enabling data-driven decision-making across the organization. Theideal candidate will possess strong technical skills, a keen understanding ofdata architecture, and a passion for optimizing data processes. Accountability Design and implement scalable and efficient data pipelines to acquire, transform, and integrate data from various sources, such as electronic health records (EHR), medical devices, claims data, and back-office enterprise data Develop data ingestion processes, including data extraction, cleansing, and validation, ensuring data quality and integrity throughout the pipeline Collaborate with cross-functional teams, including subject matter experts, analysts, and engineers, to define data requirements and ensure data pipelines meet the needs of data-driven initiatives Design and implement data integration strategies to merge disparate datasets, enabling comprehensive and holistic analysis Implement data governance practices and ensure compliance with healthcare data standards, regulations (e.g., HIPAA), and security protocols Monitor and troubleshoot pipeline and data model performance, identifying and addressing bottlenecks, and ensuring optimal system performance and data availability Design and implement data models that align with domain requirements, ensuring efficient data storage, retrieval, and delivery Apply data modeling best practices and standards to ensure consistency, scalability, and reusability of data models Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of healthcare data Develop and enforce data governance policies and procedures, including data lineage, architecture, and metadata management Collaborate with stakeholders to define data quality metrics and establish data quality improvement initiatives Document data engineering processes, methodologies, and data flows for knowledge sharing and future reference Stay up to date with emerging technologies, industry trends, and healthcare data standards to drive innovation and ensure compliance Skills 4+ years strong programming skills in object-oriented languages such as Python Proficiency in SQL Hands on experience with data integration tools, ETL/ELT frameworks, and data warehousing concepts Hands on experience with data modeling and schema design, including concepts such as star schema, snowflake schema and data normalization Familiarity with healthcare data standards (e.g., HL7, FHIR), electronic health records (EHR), medical coding systems (e.g., ICD-10, SNOMED CT), and relevant healthcare regulations (e.g., HIPAA) Hands on experience with big data processing frameworks such as Apache Hadoop, Apache Spark, etc. Working knowledge of cloud computing platforms (e.g., AWS, Azure, GCP) and related services (e.g., DMS, S3, Redshift, BigQuery) Experience integrating heterogeneous data sources, aligning data models and mapping between different data schemas Understanding of metadata management principles and tools for capturing, storing, and managing metadata associated with data models and semantic data layers Ability to track the flow of data and its transformations across data models, ensuring transparency and traceability Understanding of data governance principles, data quality management, and data security best practices Strong problem-solving and analytical skills with the ability to work with complex datasets and data integration challenges Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams Education Bachelor's orMaster's degree in computer science, information systems, or a relatedfield. Provenexperience as a Data Engineer or similar role with a focus on healthcare data.

Posted 1 week ago

Apply

5.0 - 7.0 years

5 - 9 Lacs

Chennai

Work from Office

Naukri logo

Design,develop, and maintain scalable data pipelines and systems to support thecollection, integration, and analysis of healthcare and enterprise data. Theprimary responsibilities of this role include designing and implementingefficient data pipelines, architecting robust data models, and adhering to datamanagement best practices. In this position, you will play a crucial part intransforming raw data into meaningful insights, through development of semanticdata layers, enabling data-driven decision-making across the organization. Theideal candidate will possess strong technical skills, a keen understanding ofdata architecture, and a passion for optimizing data processes. What you will do Design and implement scalable and efficient data pipelines to acquire, transform, and integrate data from various sources, such as electronic health records (EHR), medical devices, claims data, and back-office enterprise data Develop data ingestion processes, including data extraction, cleansing, and validation, ensuring data quality and integrity throughout the pipeline Collaborate with cross-functional teams, including subject matter experts, analysts, and engineers, to define data requirements and ensure data pipelines meet the needs of data-driven initiatives Design and implement data integration strategies to merge disparate datasets, enabling comprehensive and holistic analysis Implement data governance practices and ensure compliance with healthcare data standards, regulations (e.g., HIPAA), and security protocols Monitor and troubleshoot pipeline and data model performance, identifying and addressing bottlenecks, and ensuring optimal system performance and data availability Design and implement data models that align with domain requirements, ensuring efficient data storage, retrieval, and delivery Apply data modeling best practices and standards to ensure consistency, scalability, and reusability of data models Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of healthcare data Develop and enforce data governance policies and procedures, including data lineage, architecture, and metadata management Collaborate with stakeholders to define data quality metrics and establish data quality improvement initiatives Document data engineering processes, methodologies, and data flows for knowledge sharing and future reference Stay up to date with emerging technologies, industry trends, and healthcare data standards to drive innovation and ensure compliance Who you are 4+ years strong programming skills in object-oriented languages such as Python Proficiency in SQL Hands on experience with data integration tools, ETL/ELT frameworks, and data warehousing concepts Hands on experience with data modeling and schema design, including concepts such as star schema, snowflake schema and data normalization Familiarity with healthcare data standards (e.g., HL7, FHIR), electronic health records (EHR), medical coding systems (e.g., ICD-10, SNOMED CT), and relevant healthcare regulations (e.g., HIPAA) Hands on experience with big data processing frameworks such as Apache Hadoop, Apache Spark, etc. Working knowledge of cloud computing platforms (e.g., AWS, Azure, GCP) and related services (e.g., DMS, S3, Redshift, BigQuery) Experience integrating heterogeneous data sources, aligning data models and mapping between different data schemas Understanding of metadata management principles and tools for capturing, storing, and managing metadata associated with data models and semantic data layers Ability to track the flow of data and its transformations across data models, ensuring transparency and traceability Understanding of data governance principles, data quality management, and data security best practices Strong problem-solving and analytical skills with the ability to work with complex datasets and data integration challenges Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams Education Bachelor's or Master's degree in computer science, information systems, or a relatedfield. Proven experience as a Data Engineer or similar role with a focus on healthcaredata. Soft Skills: Attention to detail. Good oral and written communication skills in English language. Or Proficient in English communication, both written and verbal. Dedicated self-starter with excellent people skills. Quick learner and a go-getter. Effective time and project management. Analytical thinker and a great team player. Strong leadership, interpersonal &problem-solving skills

Posted 1 week ago

Apply

5.0 - 12.0 years

20 - 25 Lacs

Kolkata, Mumbai, New Delhi

Work from Office

Naukri logo

You have an entrepreneurial spirit You enjoy working as a part of well-knit teams You value the team over the individual You welcome diversity at work and within the greater community You aren't afraid to take risks You appreciate a growth path with your leadership team that journeys how you can grow inside and outside of the organization You thrive upon continuing education programs that your company sponsors to strengthen your skills and for you to become a thought leader ahead of the industry curve You are excited about creating change because your skills can help the greater good of every customer, industry and community We are hiring a talented GCP Lead Solution Architect (Data Migration, LSHC) Looking for someone who can drive the solution design, architecture meetings with client and mentor the team, who can lead the team, with GCP as the expertise

Posted 1 week ago

Apply

0 years

0 Lacs

Pune, Maharashtra, India

On-site

Linkedin logo

Looking for Immediate Joiners. We are primarily looking for a candidate as soon as possible with strong proficiency in the following areas: SQL, PL/SQL, and Oracle Query Development – Solid hands-on experience in writing efficient and optimized queries for both Oracle and SQL Server databases. DAX and MDX – Good proficiency in writing DAX (Data Analysis Expressions) for Power BI and MDX (Multidimensional Expressions) for SSAS. ETL & Reporting using MSBI Stack – Experience in developing, deploying, and maintaining solutions using: SSIS (SQL Server Integration Services) SSAS (SQL Server Analysis Services) SSRS (SQL Server Reporting Services) Power BI The candidate should be capable of integrating these tools with Oracle and Hadoop ecosystems (through Spark and Hive) . Agile Practices & Ceremonies – Familiarity with Agile delivery frameworks and tools such as Rally or JIRA . ITSM Processes – Experience in handling incidents, changes, and problem management through BMC Remedy . Support Tasks – Willingness to take on L1 and L2 support responsibilities related to the above platforms and solutions. Domain Knowledge – Understanding of the Payments domain is a plus. Azure Data Services – Hands-on experience with Azure data services (good to have). Show more Show less

Posted 1 week ago

Apply

3.0 years

0 Lacs

Gurugram, Haryana, India

On-site

Linkedin logo

Role Description: Sr. Data Engineer – Big Data The ideal candidate is a hands-on technology developer with experience in developing scalable applications and platforms. They must be at ease working in an agile environment with little supervision. The person should be a self-motivated person with a passion for problem solving and continuous learning. Role and responsibilities •Strong technical, analytical, and problem-solving skills •Strong organizational skills, with the ability to work autonomously as well as in a team-based environment • Data pipeline framework development Technical skills requirements The candidate must demonstrate proficiency in, •CDH On-premise for data processing and extraction •Ability to own and deliver on large, multi-faceted projects •Fluency in complex SQL and experience with RDBMSs • Project Experience in CDH experience, Spark, PySpark, Scala, Python, NiFi, Hive, NoSql DBs) Experience designing and building big data pipelines Experience working on large scale, distributed systems \ •Strong hands-on experience of programming language like PySpark, Scala with Spark, Python. Certification in Hadoop/Big Data – Hortonworks/Cloudera •Unix or Shell scripting •Strong delivery background across the delivery of high-value, business-facing technical projects in major organizations •Experience of managing client delivery teams, ideally coming from a Data Engineering / Data Science environment Job Types: Full-time, Permanent Benefits: Health insurance Provident Fund Schedule: Day shift Ability to commute/relocate: Gurugram, Haryana: Reliably commute or planning to relocate before starting work (Required) Application Question(s): Are you serving notice period at your current organization? Education: Bachelor's (Required) Experience: Python: 3 years (Required) Work Location: In person Show more Show less

Posted 1 week ago

Apply

5.0 years

0 Lacs

Kolkata, West Bengal, India

On-site

Linkedin logo

Job Location- Kolkata (Hybrid) Experience Level - 5+ Years Mandatory Skills -Azure Databricks +SQL +Pyspark Primary Roles and Responsibilities : Developing Modern Data Warehouse solutions using Databricks and Azure Stack Ability to provide solutions that are forward-thinking in data engineering and analytics space Collaborate with DW/BI leads to understand new ETL pipeline development requirements. Triage issues to find gaps in existing pipelines and fix the issues Work with business to understand the need in reporting layer and develop data model to fulfill reporting needs Help joiner team members to resolve issues and technical challenges. Drive technical discussion with client architect and team members Orchestrate the data pipelines in scheduler via Airflow Skills and Qualifications : Bachelor's and/or master’s degree in computer science or equivalent experience. Must have total 5+ yrs. of IT experience and 3+ years' experience in Data warehouse/ETL projects. Deep understanding of Star and Snowflake dimensional modelling. Strong knowledge of Data Management principles Good understanding of Databricks Data & AI platform and Databricks Delta Lake Architecture Should have hands-on experience in SQL , Python and Spark (PySpark) Candidate must have experience in Azure stack Desirable to have ETL with batch and streaming (Kinesis). Experience in building ETL / data warehouse transformation processes Experience with Apache Kafka for use with streaming data / event-based data Experience with other Open-Source big data products Hadoop (incl. Hive, Pig, Impala) Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J) Experience working with structured and unstructured data including imaging & geospatial data. Experience working in a Dev/Ops environment with tools such as Terraform, CircleCI, GIT. Proficiency in RDBMS, complex SQL, PL/SQL, Unix Shell Scripting, performance tuning and troubleshoot Databricks Certified Data Engineer Associate/Professional Certification (Desirable). Comfortable working in a dynamic, fast-paced, innovative environment with several ongoing concurrent projects Should have experience working in Agile methodology Strong verbal and written communication skills. Strong analytical and problem-solving skills with a high attention to detail. Show more Show less

Posted 1 week ago

Apply

5.0 years

0 Lacs

Bengaluru, Karnataka, India

On-site

Linkedin logo

Data scientist with strong background in data mining, machine learning, recommendation systems, and statistics. Should possess signature strengths of a qualified mathematician with ability to apply concepts of Mathematics, Applied Statistics, with specialization in one or more of NLP, Computer Vision, Speech, Data mining to develop models that provide effective solution. A strong data engineering background with hands-on coding capabilities is needed to own and deliver outcomes. A Master’s or PhD Degree in a highly quantitative field (Computer Science, Machine Learning, Operational Research, Statistics, Mathematics, etc.) or equivalent experience, 5+ years of industry experience in predictive modelling, data science and analysis, with prior experience in a ML or data scientist role and a track record of building ML or DL models. Responsibilities and skills Work with our customers to deliver a ML/DL project from beginning to end, including understanding the business need, aggregating data, exploring data, building & validating predictive models and deploying completed models to deliver business impact to organizations. Selecting features, building and optimizing classifiers using ML techniques. Data mining using state-of-the-art methods, creating text mining pipelines to clean & process large unstructured datasets to reveal high-quality information and hidden insights using machine learning techniques. Should be able to appreciate and work on: Should be able to appreciate and work on Computer Vision problems, for example, extract rich information from images to categorize and process visual data, develop machine learning algorithms for object and image classification, experience in using DBScan, PCA, Random Forests and Multinomial Logistic Regression to select the best features to classify objects. OR Deep understanding of NLP such as fundamentals of information retrieval, deep learning approaches, transformers, attention models, text summarisation, attribute extraction etc. Preferable experience in one or more of the following areas: recommender systems, moderation of user-generated content, sentiment analysis, etc. OR Speech recognition, speech to text and vice versa, understanding NLP and IR, text summarisation, statistical and deep learning approaches to text processing. Experience of having worked in these areas. Excellent understanding of machine learning techniques and algorithms, such as k-NN, Naive Bayes, SVM, Decision Forests, etc. Appreciation for deep learning frameworks like MXNet, Caffe 2, Keras, Tensorflow. Experience in working with GPUs to develop models, handling terabyte-size datasets. Experience with common data science toolkits such as R, Weka, NumPy, MatLab, mlr, mllib, Scikit learn, caret etc – excellence in at least one of these is highly desirable. Should be able to work hands-on in Python, R etc. Should closely collaborate & work with engineering teams to iteratively analyse data using Scala, Spark, Hadoop, Kafka, Storm etc. Experience with NoSQL databases and familiarity with data visualization tools will be of great advantage. What will you experience in terms of culture at Sahaj? A culture of trust, respect and transparency Opportunity to collaborate with some of the finest minds in the industry Work across multiple domains What are the benefits of being at Sahaj? Unlimited leaves Life insurance & private health insurance Stock options No hierarchy Open Salaries Show more Show less

Posted 1 week ago

Apply

Exploring Hadoop Jobs in India

The demand for Hadoop professionals in India has been on the rise in recent years, with many companies leveraging big data technologies to drive business decisions. As a job seeker exploring opportunities in the Hadoop field, it is important to understand the job market, salary expectations, career progression, related skills, and common interview questions.

Top Hiring Locations in India

  1. Bangalore
  2. Mumbai
  3. Pune
  4. Hyderabad
  5. Chennai

These cities are known for their thriving IT industry and have a high demand for Hadoop professionals.

Average Salary Range

The average salary range for Hadoop professionals in India varies based on experience levels. Entry-level Hadoop developers can expect to earn between INR 4-6 lakhs per annum, while experienced professionals with specialized skills can earn upwards of INR 15 lakhs per annum.

Career Path

In the Hadoop field, a typical career path may include roles such as Junior Developer, Senior Developer, Tech Lead, and eventually progressing to roles like Data Architect or Big Data Engineer.

Related Skills

In addition to Hadoop expertise, professionals in this field are often expected to have knowledge of related technologies such as Apache Spark, HBase, Hive, and Pig. Strong programming skills in languages like Java, Python, or Scala are also beneficial.

Interview Questions

  • What is Hadoop and how does it work? (basic)
  • Explain the difference between HDFS and MapReduce. (medium)
  • How do you handle data skew in Hadoop? (medium)
  • What is YARN in Hadoop? (basic)
  • Describe the concept of NameNode and DataNode in HDFS. (medium)
  • What are the different types of join operations in Hive? (medium)
  • Explain the role of the ResourceManager in YARN. (medium)
  • What is the significance of the shuffle phase in MapReduce? (medium)
  • How does speculative execution work in Hadoop? (advanced)
  • What is the purpose of the Secondary NameNode in HDFS? (medium)
  • How do you optimize a MapReduce job in Hadoop? (medium)
  • Explain the concept of data locality in Hadoop. (basic)
  • What are the differences between Hadoop 1 and Hadoop 2? (medium)
  • How do you troubleshoot performance issues in a Hadoop cluster? (advanced)
  • Describe the advantages of using HBase over traditional RDBMS. (medium)
  • What is the role of the JobTracker in Hadoop? (medium)
  • How do you handle unstructured data in Hadoop? (medium)
  • Explain the concept of partitioning in Hive. (medium)
  • What is Apache ZooKeeper and how is it used in Hadoop? (advanced)
  • Describe the process of data serialization and deserialization in Hadoop. (medium)
  • How do you secure a Hadoop cluster? (advanced)
  • What is the CAP theorem and how does it relate to distributed systems like Hadoop? (advanced)
  • How do you monitor the health of a Hadoop cluster? (medium)
  • Explain the differences between Hadoop and traditional relational databases. (medium)
  • How do you handle data ingestion in Hadoop? (medium)

Closing Remark

As you navigate the Hadoop job market in India, remember to stay updated on the latest trends and technologies in the field. By honing your skills and preparing diligently for interviews, you can position yourself as a strong candidate for lucrative opportunities in the big data industry. Good luck on your job search!

cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Featured Companies