8 Hadoop Stack Jobs

Title and Summary Lead Data EngineerOverview We are the global technology company behind the worlds fastest payments processing network. We are a vehicle for commerce, a connection to financial systems for the previously excluded, a technology innovation lab, and the home of Priceless. We ensure every employee has the opportunity to be a part of something bigger and to change lives. We believe as our company grows, so should you. We believe in connecting everyone to endless, priceless possibilities. Our team within Mastercard Services: The Services org is a key differentiator for Mastercard, providing the cutting-edge services that are used by some of the world's largest organizations to make multi-million-dollar decisions and grow their businesses. Focused on thinking big and scaling fast around the globe, this agile team is responsible for end-to-end solutions for a diverse global customer base. Centered on data-driven technologies and innovation, these services include payments-focused consulting, loyalty and marketing programs, business Test & Learn experimentation, and data-driven information and risk management services. Advanced Analytics Program: Within the Services Technology Team, the Advanced Analytics program is a relatively new program that is comprised of a rich set of products that provide accurate perspectives on Credit Risk, Portfolio Optimization, and Ad Insights. Currently, we are enhancing our customer experience with new user interfaces, moving to API and web application-based data publishing to allow for seamless integration in other Mastercard products and externally, utilizing new data sets and algorithms to further analytic capabilities, and generating scalable big data processes. We are looking for an innovative lead data engineer who will lead the technical design and development of an Analytic Foundation. The Analytic Foundation is a suite of individually commercialized analytical capabilities that also includes a comprehensive data platform. These services will be offered through a series of APIs that deliver data and insights from various points along a central data store. This individual will partner closely with other areas of the business to build and enhance solutions that drive value for our customers. Engineers work in small, flexible teams. Every team member contributes to designing, building, and testing features. The range of work you will encounter varies from building intuitive, responsive UIs to designing backend data models, architecting data flows, and beyond. There are no rigid organizational structures, and each team uses processes that work best for its members and projects. Here are a few examples of products in our space: Portfolio Optimizer (PO) is a solution that leverages Mastercards data assets and analytics to allow issuers to identify and increase revenue opportunities within their credit and debit portfolios. Audiences uses anonymized and aggregated transaction insights to offer targeting segments that have high likelihood to make purchases within a category to allow for more effective campaign planning and activation. Credit Risk products are a new suite of APIs and tooling to provide lenders real-time access to KPIs and insights serving thousands of clients to make smarter risk decisions using Mastercard data. Help found a new, fast-growing engineering team! Position Responsibilities: As a Lead Data Engineer within Advanced Analytics team, you will: Lead collaboration with data scientists to understand the existing modeling pipeline and identify optimization opportunities. Oversee the integration and management of data from various sources and storage systems, establishing processes and pipelines to produce cohesive datasets for analysis and modeling. Partner with Product Managers and Customer Experience Designers to develop a deep understanding of users and use cases and apply that knowledge to scoping and building new modules and features Design and develop data pipelines to automate repetitive tasks within data science and data engineering. Demonstrated experience leading cross-functional teams or working across different teams to solve complex problems. Identify patterns and innovative solutions in existing spaces, consistently seeking opportunities to simplify, automate tasks, and build reusable components for multiple use cases and teams. Create data products that are well-modeled, thoroughly documented, and easy to understand and maintain. Comfortable leading projects in environments with undefined or ambiguous requirements. Mentor junior data engineers Ideal Candidate Qualifications: High proficiency in using Python or Scala, Spark, Hadoop platforms & tools (Hive, Impala, Oozie, Airflow, NiFi, Scoop), SQL to build Big Data products & platforms Extensive experience with Spark Processing engine. Proficiency in, at least, one modern programming language such as Python, Java or Scala Strong Computer Science fundamentals in object-oriented design, data structures, algorithm design, problem solving, and complexity analysis. Ability to easily move between business, data management, and technical teams; ability to quickly intuit the business use case and identify technical solutions to enable it Working Knowledge in Software Development engineering Paradigms along with Data Engineering. Relational Databases as well as NoSQL experience Experience in cloud technologies like Databricks/AWS/Azure Basic Shell scripting and knowledge of Linux/Unix systems Experience in designing & developing software at scale Strong written and verbal English communication skills.

Title and Summary Senior Specialist, Product DevelopmentAbout Mastercard: Mastercard is a global technology company in the payments industry. Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart, and accessible. Using secure data and networks, partnerships, and passion, our innovations and solutions help individuals, financial institutions, governments, and businesses realize their greatest potential. Our decency quotient, or DQ, drives our culture and everything we do inside and outside of our company. With connections across more than 210 countries and territories, we are building a sustainable world that unlocks priceless possibilities for all. Our Team: As consumer preference for digital payments continues to grow, ensuring a seamless and secure consumer experience is top of mind. The Optimization Solutions team focuses on tracking digital performance across all products and regions, understanding the factors influencing performance and the broader industry landscape. This includes delivering data-driven insights and business recommendations, engaging directly with key external stakeholders on implementing optimization solutions (new and existing), and partnering across the organization to drive alignment and ensure action is taken. Are you excited about Data Assets and the value they bring to an organization? Are you an evangelist for data-driven decision-making? Are you motivated to be part of a team that builds large-scale Analytical Capabilities supporting end users across 6 continents? Do you want to be the go-to resource for data science & analytics in the company? The Role: Work closely with the global optimization solutions team to architect, develop, and maintain advanced reporting and data visualization capabilities on large volumes of data to support data insights and analytical needs across products, markets, and services. Focus on building solutions using Machine Learning and creating actionable insights to support product optimization and sales enablement. Prototype new algorithms, experiment, evaluate, and deliver actionable insights. Drive the evolution of products with an impact focused on data science and engineering. Design machine learning systems and self-running artificial intelligence (AI) software to automate predictive models. Perform data ingestion, aggregation, and processing on high volume and high dimensionality data to drive and enable data unification and produce relevant insights. Continuously innovate and determine new approaches, tools, techniques, and technologies to solve business problems and generate business insights and recommendations. Apply knowledge of metrics, measurements, and benchmarking to complex and demanding solutions. All About You: A superior academic record at a leading university in Computer Science, Data Science, Technology, Mathematics, Statistics, or a related field, or equivalent work experience. Experience in data management, data mining, data analytics, data reporting, data product development, and quantitative analysis. Strong analytical skills with a track record of translating data into compelling insights. Prior experience working in a product development role. Knowledge of ML frameworks, libraries, data structures, data modeling, and software architecture. Proficiency in using Python/Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), and SQL to build Big Data products & platforms. Experience with Enterprise Business Intelligence Platform/Data platform i.e., Tableau, PowerBI is a plus. Demonstrated success interacting with stakeholders to understand technical needs and ensuring analyses and solutions meet their needs effectively. Ability to build a strong narrative on the business value of products and actively participate in sales enablement efforts. Able to work in a fast-paced, deadline-driven environment as part of a team and as an individual contributor.

We are looking for an innovative software engineering lead who will lead the technical design and development of an Analytic Foundation. The Analytic Foundation is a suite of individually commercialized analytical capabilities (think prediction as a service, matching as a service or forecasting as a service) that also includes a comprehensive data platform. These services will be offered through a series of APIs that deliver data and insights from various points along a central data store. This individual will partner closely with other areas of the business to build and enhance solutions that drive value for our customers. Engineers work in small, flexible teams. Every team member contributes to designing, building, and testing features. The range of work you will encounter varies from building intuitive, responsive UIs to designing backend data models, architecting data flows, and beyond. There are no rigid organizational structures, and each team uses processes that work best for its members and projects. Here are a few examples of products in our space: Portfolio Optimizer (PO) is a solution that leverages Mastercards data assets and analytics to allow issuers to identify and increase revenue opportunities within their credit and debit portfolios. Ad Insights uses anonymized and aggregated transaction insights to offer targeting segments that have high likelihood to make purchases within a category to allow for more effective campaign planning and activation. Help found a new, fast-growing engineering team! Position Responsibilities: As a Data Engineer within DAAI, you will: Play a large role in the implementation of complex features Push the boundaries of analytics and powerful, scalable applications Build and maintain analytics and data models to enable performant and scalable products Ensure a high-quality code base by writing and reviewing performant, well-tested code Mentor junior engineers and teammates Drive innovative improvements to team development processes Partner with Product Managers and Customer Experience Designers to develop a deep understanding of users and use cases and apply that knowledge to scoping and building new modules and features Collaborate across teams with exceptional peers who are passionate about what they do Ideal Candidate Qualifications: 4+ years of data engineering experience in an agile production environment Experience leading the design and implementation of large, complex features in full-stack applications Ability to easily move between business, data management, and technical teams; ability to quickly intuit the business use case and identify technical solutions to enable it Experience leveraging open source tools, predictive analytics, machine learning, Advanced Statistics, and other data techniques to perform analyses High proficiency in using Python or Scala, Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi, Scoop), SQL to build Big Data products & platforms Experience in building and deploying production-level data-driven applications and data processing workflows/pipelines and/or implementing machine learning systems at scale in Java, Scala, or Python and deliver analytics involving all phases like data ingestion, feature engineering, modeling, tuning, evaluating, monitoring, and presenting Experience in cloud technologies like Databricks/AWS/Azure Strong technologist with proven track record of learning new technologies and frameworks Customer-centric development approach Passion for analytical quantitative problem solving Experience identifying and implementing technical improvements to development processes Collaboration skills with experience working with people across roles and geographies Motivation, creativity, self-direction, and desire to thrive on small project teams Superior academic record with a degree in Computer Science or related technical field Strong written and verbal English communication skills #AI3

Ready to build the future with AI At Genpact, we don&rsquot just keep up with technology&mdashwe set the pace. AI and digital innovation are redefining industries, and we&rsquore leading the charge. Genpact&rsquos , our industry-first accelerator, is an example of how we&rsquore scaling advanced technology solutions to help global enterprises work smarter, grow faster, and transform at scale. From large-scale models to , our breakthrough solutions tackle companies most complex challenges. If you thrive in a fast-moving, innovation-driven environment, love building and deploying cutting-edge AI solutions, and want to push the boundaries of what&rsquos possible, this is your moment. Genpact (NYSE: G) is an advanced technology services and solutions company that delivers lasting value for leading enterprises globally. Through our deep business knowledge, operational excellence, and cutting-edge solutions - we help companies across industries get ahead and stay ahead. Powered by curiosity, courage, and innovation , our teams implement data, technology, and AI to create tomorrow, today. Get to know us at and on , , , and . Inviting applications for the role of Principal Consultant- Data Engineer! Responsibilities Responsibilities (The primary tasks, functions and deliverables of the role) Design and build reusable components, frameworks and libraries at scale to support analytics products Design and implement product features in collaboration with business and Technology stakeholders Identify and solve issues concerning data management to improve data quality Clean, prepare and optimize data for ingestion and consumption Collaborate on the implementation of new data management projects and re-structure of the current data architecture Implement automated workflows and routines using workflow scheduling tools Build continuous integration, test-driven development and production deployment frameworks Analyze and profile data for designing scalable solutions Troubleshoot data issues and perform root cause analysis to proactively resolve product and operational issues Qualifications we seek in you! Minimum Qualifications Experience : Strong understanding of data structures and algorithms Strong understanding of solution and technical design Has a strong problem solving and analytical mindset. Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders Able to quickly pick up new programming languages, technologies, and frameworks Experience building cloud scalable, real time and high-performance data lake solutions Fair understanding of developing complex data solutions Experience working on end-to-end solution design Willing to learn new skills and technologies Has a passion for data solutions Preferred Qualifications/ Skills Hands on experience in Databricks and AWS - EMR [Hive, Pyspark ], S3, Athena. Familiarity with Spark Structured Streaming experience working experience with Hadoop stack dealing huge volumes of data in a scalable fashion hands-on experience with SQL, ETL, data transformation and analytics functions hands-on Python experience including Batch scripting, data manipulation, distributable packages experience working with batch orchestration tools such as Apache Airflow or equivalent, preferable Airflow working with code versioning tools such as GitHub or BitBucket expert level understanding of repo design and best practices Familiarity with deployment automation tools such as Jenkins hands-on experience designing and building ETL pipelines expert with data ingest, change data capture, data quality hand on experience with API development designing and developing relational database objects knowledgeable on logical and physical data modelling concepts some experience with Snowflake Familiarity with Tableau or Cognos use cases Why join Genpact Lead AI-first transformation - Build and scale AI solutions that redefine industries Make an impact - Drive change for global enterprises and solve business challenges that matter Accelerate your career &mdashGain hands-on experience, world-class training, mentorship, and AI certifications to advance your skills Grow with the best - Learn from top engineers, data scientists, and AI experts in a dynamic, fast-moving workplace Committed to ethical AI - Work in an environment where governance, transparency, and security are at the core of everything we build Thrive in a values-driven culture - Our courage, curiosity, and incisiveness - built on a foundation of integrity and inclusion - allow your ideas to fuel progress Come join the 140,000+ coders, tech shapers, and growth makers at Genpact and take your career in the only direction that matters: Up. Let&rsquos build tomorrow together. Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color , religion or belief, sex, age, national origin, citizenship status, marital status, military/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values respect and integrity, customer focus, and innovation. Furthermore, please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way. Examples of such scams include purchasing a %27starter kit,%27 paying to apply, or purchasing equipment or training.

Ready to build the future with AI At Genpact, we don&rsquot just keep up with technology&mdashwe set the pace. AI and digital innovation are redefining industries, and we&rsquore leading the charge. Genpact&rsquos , our industry-first accelerator, is an example of how we&rsquore scaling advanced technology solutions to help global enterprises work smarter, grow faster, and transform at scale. From large-scale models to , our breakthrough solutions tackle companies most complex challenges. If you thrive in a fast-moving, innovation-driven environment, love building and deploying cutting-edge AI solutions, and want to push the boundaries of what&rsquos possible, this is your moment. Genpact (NYSE: G) is an advanced technology services and solutions company that delivers lasting value for leading enterprises globally. Through our deep business knowledge, operational excellence, and cutting-edge solutions - we help companies across industries get ahead and stay ahead. Powered by curiosity, courage, and innovation , our teams implement data, technology, and AI to create tomorrow, today. Get to know us at and on , , , and . Inviting applications for the role of Lead Consultant- Data Engineer! Responsibilities Responsibilities (The primary tasks, functions and deliverables of the role) Design and build reusable components, frameworks and libraries at scale to support analytics products Design and implement product features in collaboration with business and Technology stakeholders Identify and solve issues concerning data management to improve data quality Clean, prepare and optimize data for ingestion and consumption Collaborate on the implementation of new data management projects and re-structure of the current data architecture Implement automated workflows and routines using workflow scheduling tools Build continuous integration, test-driven development and production deployment frameworks Analyze and profile data for designing scalable solutions Troubleshoot data issues and perform root cause analysis to proactively resolve product and operational issues Qualifications we seek in you! Minimum Qualifications Experience : Strong understanding of data structures and algorithms Strong understanding of solution and technical design Has a strong problem solving and analytical mindset. Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders Able to quickly pick up new programming languages, technologies, and frameworks Experience building cloud scalable, real time and high-performance data lake solutions Fair understanding of developing complex data solutions Experience working on end-to-end solution design Willing to learn new skills and technologies Has a passion for data solutions Preferred Qualifications/ Skills Hands on experience in Databricks and AWS - EMR [Hive, Pyspark ], S3, Athena. Familiarity with Spark Structured Streaming experience working experience with Hadoop stack dealing huge volumes of data in a scalable fashion hands-on experience with SQL, ETL, data transformation and analytics functions hands-on Python experience including Batch scripting, data manipulation, distributable packages experience working with batch orchestration tools such as Apache Airflow or equivalent, preferable Airflow working with code versioning tools such as GitHub or BitBucket expert level understanding of repo design and best practices Familiarity with deployment automation tools such as Jenkins hands-on experience designing and building ETL pipelines expert with data ingest, change data capture, data quality hand on experience with API development designing and developing relational database objects knowledgeable on logical and physical data modelling concepts some experience with Snowflake Familiarity with Tableau or Cognos use cases Why join Genpact Lead AI-first transformation - Build and scale AI solutions that redefine industries Make an impact - Drive change for global enterprises and solve business challenges that matter Accelerate your career &mdashGain hands-on experience, world-class training, mentorship, and AI certifications to advance your skills Grow with the best - Learn from top engineers, data scientists, and AI experts in a dynamic, fast-moving workplace Committed to ethical AI - Work in an environment where governance, transparency, and security are at the core of everything we build Thrive in a values-driven culture - Our courage, curiosity, and incisiveness - built on a foundation of integrity and inclusion - allow your ideas to fuel progress Come join the 140,000+ coders, tech shapers, and growth makers at Genpact and take your career in the only direction that matters: Up. Let&rsquos build tomorrow together. Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color , religion or belief, sex, age, national origin, citizenship status, marital status, military/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values respect and integrity, customer focus, and innovation. Furthermore, please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way. Examples of such scams include purchasing a %27starter kit,%27 paying to apply, or purchasing equipment or training.

As an experienced professional, you will be responsible for implementing and supporting Hadoop platform-based applications to efficiently store, retrieve, and process terabytes of data. Your expertise will contribute to the seamless functionality of the system. To excel in this role, you should possess 4+ years of core Java or 2+ years of Python experience. Additionally, having at least 1+ years of working experience with the Hadoop stack including HDFS, MapReduce, HBase, and other related technologies is desired. A solid background of 1+ years in database management with MySQL or equivalent systems will be an added advantage. If you meet the qualifications and are enthusiastic about this opportunity, we encourage you to share your latest CV with us at data@scalein.com or reach out to us via our contact page. Your contribution will be pivotal in driving the success of our data management initiatives.,

Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose - the relentless pursuit of a world that works better for people - we serve and transform leading enterprises, including the Fortune Global 500, with our deep business and industry knowledge, digital operations services, and expertise in data, technology, and AI. Inviting applications for the role of Principal Consultant Big Data! In this role, as a trusted advisor to the business, establish partnerships, assess business needs, provide critical interpretation of data, form hypotheses and synthesize conclusions into recommendations, turning data analysis into insights Responsibilities: Design, develop, and maintain scalable data processing solutions using Python/Scala and Spark per business needs • Implement and manage CI/CD pipelines to automate the deployment process. • Hands on experience with Cribl an added advantage. • Perform production bug fixes to maintain system stability and reliability. • Develop scalable microservices to be deployed on Hybrid cloud platforms like GCP Anthos • Strong understanding of the Hadoop ecosystem, including HDFS, Hive, Hbase and other related technologies. • Stay updated with the latest industry trends and technologies in big data and cloud computing. Qualifications we seek in you! Minimum qualifications: Bachelor's/Graduation/Equivalent: BE/B- Tech, MCA, MBA • Excellent Communication skill and effectively interact with business user • Ability to interact with business as well as technical teams • Ability to learn and respond quickly to the fast-changing business environment • Ability to multitask and excellent interpersonal skills • Analytical mindset with a problem-solving approach Preferred qualifications: • Very good written and presentation / verbal communication skills with experience of customer interfacing role. • In-depth requirement understanding skills with good analytical and problem-solving ability, interpersonal efficiency, and positive attitude. • Good knowledge of writing complex SQL • Ability to work in an agile environment with a focus on continuous improvement • Self-motivated and eager to learn • A team player and able to lead and initiate • Experience in the telecommunication industry • Experience with cloud providers (e.g., AWS) Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color, religion or belief, sex, age, national origin, citizenship status, marital status, military/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values diversity and inclusion, respect and integrity, customer focus, and innovation. Get to know us at www.genpact.com and on X, Facebook, LinkedIn, and YouTube. Furthermore, please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way. Examples of such scams include purchasing a 'starter kit,' paying to apply, or purchasing equipment or training.

Wipro Limited (NYSEWIT, BSE507685, NSEWIPRO) is a leading technology services and consulting company focused on building innovative solutions that address clients’ most complex digital transformation needs. Leveraging our holistic portfolio of capabilities in consulting, design, engineering, and operations, we help clients realize their boldest ambitions and build future-ready, sustainable businesses. With over 230,000 employees and business partners across 65 countries, we deliver on the promise of helping our customers, colleagues, and communities thrive in an ever-changing world. For additional information, visit us at www.wipro.com. About The Role Role: Service Desk Manager ? Band C1 – Role (Data Architect) Location – Chennai, Noida Total exp – 11+ Years The candidate must have overall 11+ years of experience in ETL and Data Warehouse of which 3-4 years on Hadoop platform and at least 2 year in Cloud Big Data Environment. Must have hands on experience on Hadoop services like HIVE/ Spark/ Scala /Sqoop Must have hands on in writing complex use case driven SQLs Should have about 3+ years of hands-on good knowledge of AWS Cloud and On-Prem related key services and concepts. Should have 3+ years of working experience with AWS Cloud tools like EMR, Redshift, Glue S3 Should have been involved in On-Prem to Cloud Migration process. Should have good knowledge with HIVE / Spark / Scala scripts Should have good knowledge on Unix Shell scripting Should be flexible to overlap US business hours Should be able to drive technical design on Cloud applications Should be able to guide & drive the team members for cloud implementations Should be well versed with the costing model and best practices of the services to be used for Data Processing Pipelines in Cloud Environment. AWS Certified applicants preferable ? ? ? Competencies Client Centricity Passion for Results Collaborative Working Problem Solving & Decision Making Effective communication Reinvent your world. We are building a modern Wipro. We are an end-to-end digital transformation partner with the boldest ambitions. To realize them, we need people inspired by reinvention. Of yourself, your career, and your skills. We want to see the constant evolution of our business and our industry. It has always been in our DNA - as the world around us changes, so do we. Join a business powered by purpose and a place that empowers you to design your own reinvention. Come to Wipro. Realize your ambitions. Applications from people with disabilities are explicitly welcome.