Get alerts for new jobs matching your selected skills, preferred locations, and experience range. Manage Job Alerts
5.0 - 8.0 years
1 - 2 Lacs
Gurugram
Work from Office
Urgent Requirement for Tech Consultant Data Engg with 5+yrs Exp Strong knowledge of Big Data technologies (Hadoop, Spark, Snowflake, Databricks, Airflow, AWS), Python, SQL & cloud platforms (AWS, Azure)
Posted 1 week ago
5.0 - 10.0 years
5 - 9 Lacs
Bengaluru
Work from Office
PositionSenior Data Engineer - Airflow, PLSQL Experience5+ Years LocationBangalore/Hyderabad/Pune Seeking a Senior Data Engineer with strong expertise in Apache Airflow and Oracle PL/SQL, along with working experience in Snowflake and Agile methodologies. The ideal candidate will also take up Scrum Master responsibilities and lead a data engineering scrum team to deliver robust, scalable data solutions. Key Responsibilities: Design, develop, and maintain scalable data pipelines using Apache Airflow. Write and optimize complex PL/SQL queries, procedures, and packages on Oracle databases. Collaborate with cross-functional teams to design efficient data models and integration workflows. Work with Snowflake for data warehousing and analytics use cases. Own the delivery of sprint goals, backlog grooming, and facilitation of agile ceremonies as the Scrum Master. Monitor pipeline health and troubleshoot production data issues proactively. Ensure code quality, documentation, and best practices across the team. Mentor junior data engineers and promote a culture of continuous improvement. Required Skills and Qualifications: 5+ years of experience as a Data Engineer in enterprise environments. Strong expertise in Apache Airflow for orchestrating workflows. Expert in Oracle PL/SQL - stored procedures, performance tuning, debugging. Hands-on experience with Snowflake - data modeling, SQL, optimization. Working knowledge of version control (Git) and CI/CD practices. Prior experience or certification as a Scrum Master is highly desirable. Strong analytical and problem-solving skills with attention to detail. Excellent communication and leadership skills.
Posted 1 week ago
4.0 - 9.0 years
8 - 12 Lacs
Pune
Work from Office
Amdocs helps those who build the future to make it amazing. With our market-leading portfolio of software products and services, we unlock our customers innovative potential, empowering them to provide next-generation communication and media experiences for both the individual end user and enterprise customers. Our employees around the globe are here to accelerate service providers migration to the cloud, enable them to differentiate in the 5G era, and digitalize and automate their operations. Listed on the NASDAQ Global Select Market, Amdocs had revenue of $5.00 billion in fiscal 2024. For more information, visit www.amdocs.com In one sentence We are seeking a Data Engineer with advanced expertise in Databricks SQL, PySpark, Spark SQL, and workflow orchestration using Airflow. The successful candidate will lead critical projects, including migrating SQL Server Stored Procedures to Databricks Notebooks, designing incremental data pipelines, and orchestrating workflows in Azure Databricks What will your job look like Migrate SQL Server Stored Procedures to Databricks Notebooks, leveraging PySpark and Spark SQL for complex transformations. Design, build, and maintain incremental data load pipelines to handle dynamic updates from various sources, ensuring scalability and efficiency. Develop robust data ingestion pipelines to load data into the Databricks Bronze layer from relational databases, APIs, and file systems. Implement incremental data transformation workflows to update silver and gold layer datasets in near real-time, adhering to Delta Lake best practices. Integrate Airflow with Databricks to orchestrate end-to-end workflows, including dependency management, error handling, and scheduling. Understand business and technical requirements, translating them into scalable Databricks solutions. Optimize Spark jobs and queries for performance, scalability, and cost-efficiency in a distributed environment. Implement robust data quality checks, monitoring solutions, and governance frameworks within Databricks. Collaborate with team members on Databricks best practices, reusable solutions, and incremental loading strategies All you need is... Bachelor s degree in computer science, Information Systems, or a related discipline. 4+ years of hands-on experience with Databricks, including expertise in Databricks SQL, PySpark, and Spark SQL. Proven experience in incremental data loading techniques into Databricks, leveraging Delta Lake's features (e.g., time travel, MERGE INTO). Strong understanding of data warehousing concepts, including data partitioning, and indexing for efficient querying. Proficiency in T-SQL and experience in migrating SQL Server Stored Procedures to Databricks. Solid knowledge of Azure Cloud Services, particularly Azure Databricks and Azure Data Lake Storage. Expertise in Airflow integration for workflow orchestration, including designing and managing DAGs. Familiarity with version control systems (e.g., Git) and CI/CD pipelines for data engineering workflows. Excellent analytical and problem-solving skills with a focus on detail-oriented development. Preferred Qualifications Advanced knowledge of Delta Lake optimizations, such as compaction, Z-ordering, and vacuuming. Experience with real-time streaming data pipelines using tools like Kafka or Azure Event Hubs. Familiarity with advanced Airflow features, such as SLA monitoring and external task dependencies. Certifications such as Databricks Certified Associate Developer for Apache Spark or equivalent. Experience in Agile development methodologie Why you will love this job: You will be able to use your specific insights to lead business change on a large scale and drive transformation within our organization. You will be a key member of a global, dynamic and highly collaborative team with various possibilities for personal and professional development. You will have the opportunity to work in multinational environment for the global market leader in its field! We offer a wide range of stellar benefits including health, dental, vision, and life insurance as well as paid time off, sick time, and parental leave!
Posted 1 week ago
5.0 - 10.0 years
10 - 15 Lacs
Chennai, Bengaluru
Work from Office
job requisition idJR1027452 Overall Responsibilities: Data Pipeline Development: Design, develop, and maintain highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform, ensuring data integrity and accuracy. Data Ingestion: Implement and manage data ingestion processes from a variety of sources (e.g., relational databases, APIs, file systems) to the data lake or data warehouse on CDP. Data Transformation and Processing: Use PySpark to process, cleanse, and transform large datasets into meaningful formats that support analytical needs and business requirements. Performance Optimization: Conduct performance tuning of PySpark code and Cloudera components, optimizing resource utilization and reducing runtime of ETL processes. Data Quality and Validation: Implement data quality checks, monitoring, and validation routines to ensure data accuracy and reliability throughout the pipeline. Automation and Orchestration: Automate data workflows using tools like Apache Oozie, Airflow, or similar orchestration tools within the Cloudera ecosystem. Monitoring and Maintenance: Monitor pipeline performance, troubleshoot issues, and perform routine maintenance on the Cloudera Data Platform and associated data processes. Collaboration: Work closely with other data engineers, analysts, product managers, and other stakeholders to understand data requirements and support various data-driven initiatives. Documentation: Maintain thorough documentation of data engineering processes, code, and pipeline configurations. Software : Advanced proficiency in PySpark, including working with RDDs, DataFrames, and optimization techniques. Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase. Knowledge of data warehousing concepts, ETL best practices, and experience with SQL-based tools (e.g., Hive, Impala). Familiarity with Hadoop, Kafka, and other distributed computing tools. Experience with Apache Oozie, Airflow, or similar orchestration frameworks. Strong scripting skills in Linux. Category-wise Technical Skills: PySpark: Advanced proficiency in PySpark, including working with RDDs, DataFrames, and optimization techniques. Cloudera Data Platform: Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase. Data Warehousing: Knowledge of data warehousing concepts, ETL best practices, and experience with SQL-based tools (e.g., Hive, Impala). Big Data Technologies: Familiarity with Hadoop, Kafka, and other distributed computing tools. Orchestration and Scheduling: Experience with Apache Oozie, Airflow, or similar orchestration frameworks. Scripting and Automation: Strong scripting skills in Linux. Experience: 5-12 years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform. Proven track record of implementing data engineering best practices. Experience in data ingestion, transformation, and optimization on the Cloudera Data Platform. Day-to-Day Activities: Design, develop, and maintain ETL pipelines using PySpark on CDP. Implement and manage data ingestion processes from various sources. Process, cleanse, and transform large datasets using PySpark. Conduct performance tuning and optimization of ETL processes. Implement data quality checks and validation routines. Automate data workflows using orchestration tools. Monitor pipeline performance and troubleshoot issues. Collaborate with team members to understand data requirements. Maintain documentation of data engineering processes and configurations. Qualifications: Bachelors or Masters degree in Computer Science, Data Engineering, Information Systems, or a related field. Relevant certifications in PySpark and Cloudera technologies are a plus. Soft Skills: Strong analytical and problem-solving skills. Excellent verbal and written communication abilities. Ability to work independently and collaboratively in a team environment. Attention to detail and commitment to data quality.
Posted 1 week ago
0.0 - 3.0 years
0 Lacs
India
On-site
Description GroundTruth is an advertising platform that turns real-world behavior into marketing that drives in-store visits and other real business results. We use observed real-world consumer behavior, including location and purchase data, to create targeted advertising campaigns across all screens, measure how consumers respond, and uncover unique insights to help optimize ongoing and future marketing efforts. With this focus on media, measurement, and insights, we provide marketers with tools to deliver media campaigns that drive measurable impact, such as in-store visits, sales, and more. Learn more at groundtruth.com. We believe that innovative technology starts with the best talent and have been ranked one of Ad Age’s Best Places to Work in 2021, 2022, 2023 & 2025! Learn more about the perks of joining our team here. A Bit About Team GroundTruth seeks a Data Engineering Associate Software Engineer to join our Integration team. The Integration Team connects and consolidates data pipelines across Avails & Inventory Forecast, Identity Graph, and POS Integration systems to ensure accurate, timely insights. We engineer seamless data flows that fuel reliable analytics and decision-making using big data technologies, such as MapReduce, Spark, and Glue. We take pride in building an Engineering Team composed of strong communicators who collaborate with multiple business and engineering stakeholders to find compromises and solutions. Our engineers are organised and detail-oriented team players who are problem solvers with a maker mindset. As an Associate Software Engineer (ASE) on our Integration Team, you will build solutions that add new capabilities to our platform. You Will Create and maintain various data pipelines for the GroundTruth platform. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies. Work with stakeholders, including the Product, Analytics and Client Services teams, to assist with data-related technical issues and support their data infrastructure needs. Prepare detailed specifications and low-level design. Participate in code reviews. Test the product in controlled, real situations before going live. Maintain the application once it is live. Contribute ideas to improve the location platform. You Have B.Tech./B.E./M.Tech./MCA or equivalent in computer science 0-3 years of experience in Data Engineering Experience with AWS Stack used for Data engineering EC2, S3, Athena, Redshift, EMR, ECS, Lambda, and Step functions Experience in Hadoop, MapReduce, Pig, Spark, and Glue Hands-on experience with Java/Python for the orchestration of data pipelines and Data engineering tasks Experience in writing analytical queries using SQL Experience in Airflow Experience in Docker Proficient in Git How can you impress us? Knowledge of REST APIs Any experience with big data technologies like Hadoop, MapReduce, and Pig is a plus Knowledge of shell scripting. Experience with BI tools like Looker. Experience with DB maintenance Experience with Amazon Web Services and Docker Configuration management and QA practices Benefits At GroundTruth, we want our employees to be comfortable with their benefits so they can focus on doing the work they love. Parental leave- Maternity and Paternity Flexible Time Offs (Earned Leaves, Sick Leaves, Birthday leave, Bereavement leave & Company Holidays) In Office Daily Catered Breakfast, Lunch, Snacks and Beverages Health cover for any hospitalization. Covers both nuclear family and parents Tele-med for free doctor consultation, discounts on health checkups and medicines Wellness/Gym Reimbursement Pet Expense Reimbursement Childcare Expenses and reimbursements Employee referral program Education reimbursement program Skill development program Cell phone reimbursement (Mobile Subsidy program). Internet reimbursement/Postpaid cell phone bill/or both. Birthday treat reimbursement Employee Provident Fund Scheme offering different tax saving options such as Voluntary Provident Fund and employee and employer contribution up to 12% Basic Creche reimbursement Co-working space reimbursement National Pension System employer match Meal card for tax benefit Special benefits on salary account Show more Show less
Posted 1 week ago
5.0 - 9.0 years
12 - 20 Lacs
Bengaluru
Work from Office
Exprence 5-8 Years Location - Bangalore Mode C2H Hands on data engineering experience. Hands on experience with Python programming Hands-on Experience with AWS & EKS Working knowledge of Unix, Databases, SQL Working Knowledge on Databricks Working Knowledge on Airflow and DBT
Posted 1 week ago
6.0 - 8.0 years
10 - 12 Lacs
Hyderabad
Work from Office
Job Description: We are looking for a highly experienced and dynamic Senior Data Manager / Lead to oversee a team of Data Engineers and Data Scientists. This role demands a strong background in data platforms such as Snowflake and proficiency in Python , combined with excellent people management and project leadership skills . While hands-on experience in the technologies is beneficial, the primary focus of this role is on team leadership, strategic planning , and project delivery . Job Title : Senior Data Manager / Lead Location: Hyderabad (Work From Office) Shift Timing: 10AM-7PM Key Responsibilities : Lead, mentor, and manage a team of Data Engineers and Data Scientists. Oversee the design and implementation of data pipelines and analytics solutions using Snowflake and Python. Collaborate with cross-functional teams (product, business, engineering) to align data solutions with business goals. Ensure timely delivery of projects, with high quality and performance. Conduct performance reviews, training plans, and support career development for the team. Set priorities, allocate resources, and manage workloads within the data team. Drive adoption of best practices in data management, governance, and documentation. Evaluate new tools and technologies relevant to data engineering and data science. Required Skills & Qualifications: 6+ years of experience in data-related roles, with at least 23 years in a leadership or management position. Strong understanding of Snowflake architecture, performance tuning, data sharing, security, etc. Solid knowledge of Python for data engineering or data science tasks. Experience in leading data migration , ETL/ELT , and analytics projects. Ability to translate business requirements into technical solutions. Excellent leadership, communication, and stakeholder management skills. Exposure to tools like Databricks , Dataiku , Airflow , or similar platforms is a plus. Bachelors or Master’s degree in Computer Science, Engineering, Mathematics, or a related field.
Posted 1 week ago
5.0 years
0 Lacs
Trivandrum, Kerala, India
On-site
Equifax is seeking creative, high-energy and driven software engineers with hands-on development skills to work on a variety of meaningful projects. Our software engineering positions provide you the opportunity to join a team of talented engineers working with leading-edge technology. You are ideal for this position if you are a forward-thinking, committed, and enthusiastic software engineer who is passionate about technology. What You’ll Do Design, develop, and operate high scale applications across the full engineering stack Design, develop, test, deploy, maintain, and improve software. Apply modern software development practices (serverless computing, microservices architecture, CI/CD, infrastructure-as-code, etc.) Work across teams to integrate our systems with existing internal systems, Data Fabric, CSA Toolset. Participate in technology roadmap and architecture discussions to turn business requirements and vision into reality. Participate in a tight-knit, globally distributed engineering team. Triage product or system issues and debug/track/resolve by analyzing the sources of issues and the impact on network, or service operations and quality. Manage sole project priorities, deadlines, and deliverables. Research, create, and develop software applications to extend and improve on Equifax Solutions Collaborate on scalability issues involving access to data and information. Actively participate in Sprint planning, Sprint Retrospectives, and other team activity What Experience You Need Bachelor's degree or equivalent experience 5+ years of software engineering experience 5+ years experience writing, debugging, and troubleshooting code in mainstream Java, SpringBoot, TypeScript/JavaScript, HTML, CSS 5+ years experience with Cloud technology: GCP, AWS, or Azure 5+ years experience designing and developing cloud-native solutions 5+ years experience designing and developing microservices using Java, SpringBoot, GCP SDKs, GKE/Kubernetes 5+ years experience deploying and releasing software using Jenkins CI/CD pipelines, understand infrastructure-as-code concepts, Helm Charts, and Terraform constructs What could set you apart Self-starter that identifies/responds to priority shifts with minimal supervision. Experience designing and developing big data processing solutions using Dataflow/Apache Beam, Bigtable, BigQuery, PubSub, GCS, Composer/Airflow, and others UI development (e.g. HTML, JavaScript, Angular and Bootstrap) Experience with backend technologies such as JAVA/J2EE, SpringBoot, SOA and Microservices Source code control management systems (e.g. SVN/Git, Github) and build tools like Maven & Gradle. Agile environments (e.g. Scrum, XP) Relational databases (e.g. SQL Server, MySQL) Atlassian tooling (e.g. JIRA, Confluence, and Github) Developing with modern JDK (v1.7+) Automated Testing: JUnit, Selenium, LoadRunner, SoapUI Show more Show less
Posted 1 week ago
2.0 years
0 Lacs
Trivandrum, Kerala, India
On-site
What You’ll Do Design, develop, and operate high scale applications across the full engineering stack. Design, develop, test, deploy, maintain, and improve software. Apply modern software development practices (serverless computing, microservices architecture, CI/CD, infrastructure-as-code, etc.) Work across teams to integrate our systems with existing internal systems, Data Fabric, CSA Toolset. Participate in technology roadmap and architecture discussions to turn business requirements and vision into reality. Participate in a tight-knit, globally distributed engineering team. Triage product or system issues and debug/track/resolve by analyzing the sources of issues and the impact on network, or service operations and quality. Research, create, and develop software applications to extend and improve on Equifax Solutions. Manage sole project priorities, deadlines, and deliverables. Collaborate on scalability issues involving access to data and information. Actively participate in Sprint planning, Sprint Retrospectives, and other team activity What Experience You Need Bachelor's degree or equivalent experience 2+ years of software engineering experience 2+ years experience writing, debugging, and troubleshooting code in mainstream Java, SpringBoot, TypeScript/JavaScript, HTML, CSS 2+ years experience with Cloud technology: GCP, AWS, or Azure 5+ years experience designing and developing cloud-native solutions 2+ years experience designing and developing microservices using Java, Spring Framework, GCP SDKs, GKE/Kubernetes 2+ years experience deploying and releasing software using Jenkins CI/CD pipelines, understand infrastructure-as-code concepts, Helm Charts, and Terraform constructs Big Data Technologies : Spark/Scala/Hadoop What could set you apart Experience designing and developing big data processing solutions using DataProc, Dataflow/Apache Beam, Bigtable, BigQuery, PubSub, GCS, Composer/Airflow, and others Cloud Certification especially in GCP Self-starter that identifies/responds to priority shifts with minimal supervision. You have excellent leadership and motivational skills You have an inquisitive and innovative mindset with a shown ability to recognize opportunities to create distinctive value You can successfully evaluate workload to drive efficiency Show more Show less
Posted 1 week ago
15.0 years
0 Lacs
Mumbai, Maharashtra, India
On-site
Introduction A career in IBM Consulting is rooted by long-term relationships and close collaboration with clients across the globe. You'll work with visionaries across multiple industries to improve the hybrid cloud and AI journey for the most innovative and valuable companies in the world. Your ability to accelerate impact and make meaningful change for your clients is enabled by our strategic partner ecosystem and our robust technology platforms across the IBM portfolio; including Software and Red Hat. Curiosity and a constant quest for knowledge serve as the foundation to success in IBM Consulting. In your role, you'll be encouraged to challenge the norm, investigate ideas outside of your role, and come up with creative solutions resulting in ground breaking impact for a wide network of clients. Our culture of evolution and empathy centers on long-term career growth and development opportunities in an environment that embraces your unique skills and experience Your Role And Responsibilities Location : Mumbai Role Overview As a Big Data Engineer, you'll design and build robust data pipelines on Cloudera using Spark (Scala/PySpark) for ingestion, transformation, and processing of high-volume data from banking systems. Key Responsibilities Build scalable batch and real-time ETL pipelines using Spark and Hive Integrate structured and unstructured data sources Perform performance tuning and code optimization Support orchestration and job scheduling (NiFi, Airflow) Preferred Education Master's Degree Required Technical And Professional Expertise Experience: 3–15 years Proficiency in PySpark/Scala with Hive/Impala Experience with data partitioning, bucketing, and optimization Familiarity with Kafka, Iceberg, NiFi is a must Knowledge of banking or financial datasets is a plus Show more Show less
Posted 1 week ago
3.0 - 5.0 years
8 - 12 Lacs
Hyderabad
Work from Office
Data Engineer openings at Advantum Health Pvt Ltd, Hyderabad. Overview: We are looking for a Data Engineer to build and optimize robust data pipelines that support AI and RCM analytics. This role involves integrating structured and unstructured data from diverse healthcare systems into scalable, AI-ready datasets. Key Responsibilities: Design, implement, and optimize data pipelines for ingesting and transforming healthcare and RCM data. Build data marts and warehouses to support analytics and machine learning. Ensure data quality, lineage, and governance across AI use cases. Integrate data from EMRs, billing platforms, claims databases, and third-party APIs. Support data infrastructure in a HIPAA-compliant cloud environment. Qualifications: Bachelors in Computer Science, Data Engineering, or related field. 3+ years of experience with ETL/ELT pipelines using tools like Apache Airflow, dbt, or Azure Data Factory. Strong SQL and Python skills. Experience with healthcare data standards (HL7, FHIR, X12) preferred. Familiarity with data lake house architectures and AI integration best practices Ph: 9177078628 Email id: jobs@advantumhealth.com Address: Advantum Health Private Limited, Cyber gateway, Block C, 4th floor Hitech City, Hyderabad. Do follow us on LinkedIn, Facebook, Instagram, YouTube and Threads Advantum Health LinkedIn Page: https://lnkd.in/gVcQAXK3 Advantum Health Facebook Page: https://lnkd.in/g7ARQ378 Advantum Health Instagram Page: https://lnkd.in/gtQnB_Gc Advantum Health India YouTube link: https://lnkd.in/g_AxPaPp Advantum Health Threads link: https://lnkd.in/gyq73iQ6
Posted 1 week ago
2.0 years
0 Lacs
Chennai, Tamil Nadu, India
On-site
Job Title: GCP Data Engineer Location: Chennai, India Job type: FTE Mandatory Skills: Google Cloud Platform - Biq Query, Data Flow, Dataproc, Data Fusion, TERRAFORM, Tekton,Cloud SQL, AIRFLOW, POSTGRES, Airflow PySpark, Python, API Job Description 2+Years in GCP Services, Biq Query, Data Flow, Dataproc, DataPlex,DataFusion, Terraform, Tekton, Cloud SQL, Redis Memory, Airflow, Cloud Storage 2+ Years inData Transfer Utilities 2+ Years in Git / any other version control tool 2+ Years in Confluent Kafka 1+ Years of Experience in API Development 2+ Years in Agile Framework 4+ years of strong experience in python, Pyspark development. 4+ years of shell scripting to develop the adhoc jobsfor data importing/exporting Show more Show less
Posted 1 week ago
0 years
0 Lacs
Gurgaon, Haryana, India
On-site
Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title And Summary Data Scientist Who is Mastercard? Mastercard is a global technology company in the payments industry. Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart, and accessible. Using secure data and networks, partnerships, and passion, our innovations and solutions help individuals, financial institutions, governments, and businesses realize their greatest potential. Our decency quotient, or DQ, drives our culture and everything we do inside and outside of our company. With connections across more than 210 countries and territories, we are building a sustainable world that unlocks priceless possibilities for all. Our Team As consumer preference for digital payments continues to grow, ensuring a seamless and secure consumer experience is top of mind. Optimization Soltions team focuses on tracking of digital performance across all products and regions, understanding the factors influencing performance and the broader industry landscape. This includes delivering data-driven insights and business recommendations, engaging directly with key external stakeholders on implementing optimization solutions (new and existing), and partnering across the organization to drive alignment and ensure action is taken. Are you excited about Data Assets and the value they bring to an organization? Are you an evangelist for data-driven decision-making? Are you motivated to be part of a team that builds large-scale Analytical Capabilities supporting end users across 6 continents? Do you want to be the go-to resource for data science & analytics in the company? The Role Work closely with global optimization solutions team to architect, develop, and maintain advanced reporting and data visualization capabilities on large volumes of data to support data insights and analytical needs across products, markets, and services The candidate for this position will focus on Building solutions using Machine Learning and creating actionable insights to support product optimization and sales enablement. Prototype new algorithms, experiment, evaluate and deliver actionable insights. Drive the evolution of products with an impact focused on data science and engineering. Designing machine learning systems and self-running artificial intelligence (AI) software to automate predictive models. Perform data ingestion, aggregation, and processing on high volume and high dimensionality data to drive and enable data unification and produce relevant insights. Continuously innovate and determine new approaches, tools, techniques & technologies to solve business problems and generate business insights & recommendations. Apply knowledge of metrics, measurements, and benchmarking to complex and demanding solutions. All About You A superior academic record at a leading university in Computer Science, Data Science, Technology, mathematics, statistics, or a related field or equivalent work experience Experience in data management, data mining, data analytics, data reporting, data product development and quantitative analysis Strong analytical skills with track record of translating data into compelling insights Prior experience working in a product development role. knowledge of ML frameworks, libraries, data structures, data modeling, and software architecture. proficiency in using Python/Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), and SQL to build Big Data products & platforms Experience with Enterprise Business Intelligence Platform/Data platform i.e. Tableau, PowerBI is a plus. Demonstrated success interacting with stakeholders to understand technical needs and ensuring analyses and solutions meet their needs effectively. Ability to build a strong narrative on the business value of products and actively participate in sales enablement efforts. Able to work in a fast-paced, deadline-driven environment as part of a team and as an individual contributor. Corporate Security Responsibility All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must: Abide by Mastercard’s security policies and practices; Ensure the confidentiality and integrity of the information being accessed; Report any suspected information security violation or breach, and Complete all periodic mandatory security trainings in accordance with Mastercard’s guidelines. R-250830 Show more Show less
Posted 1 week ago
8.0 years
0 Lacs
Hyderabad, Telangana, India
On-site
ABOUT THE TEAM Join our dynamic team of expert engineers at Creditsafe, where we are revolutionizing the data ecosystem through strategic innovation and cutting-edge architectural modernization. As a System & Product Architect , you will lead the transformation of our systems and product architecture on AWS, managing billions of data objects with daily increments exceeding 25 million. Your expertise will be pivotal in ensuring high availability, data integrity, and outstanding performance, powering our APIs and file delivery systems to deliver seamless data experiences to our global clients. Be at the forefront of data innovation and make an impact on a global scale. ABOUT THE ROLE This role places you at the center of Creditsafe's transformation journey. You will define architectural standards, design patterns, and technical roadmaps that guide our shift to a modern cloud infrastructure. Collaborating with technologies such as Python, Linux, Airflow, AWS DynamoDB, S3, Glue, Athena, Redshift, Lambda, API Gateway, Terraform, and CI/CD pipelines, you will ensure our platform is scalable, resilient, and ready for the future. KEY DUTIES AND RESPONSIBILITIES Drive the technical vision, architecture, and design principles for system replatforming and migration. Design scalable, distributed architecture patterns that optimize for throughput, resilience, and maintainability. Create and maintain system architecture documentation, including diagrams, data flows, and design decisions. Establish governance frameworks for technical debt management and architectural compliance. Design event-driven architectures for distributed data processing using AWS technologies. Work with team to support & build APIs capable of supporting 1000+ transactions per second. Mentor engineers on architectural best practices and system design principles. Partner with security teams to ensure architectures meet compliance requirements. Contribute to technical roadmap aligned with company’s vision & Product roadmap. SKILLS AND QUALIFICATIONS 8+ years of software engineering experience, with at least 4 years in system architecture. Proven track record in large-scale replatforming and system modernization initiatives. Cloud-native architecture expertise, particularly with AWS services (Redshift, S3, DynamoDB, Lambda, API Gateway). Solid understanding of data platforms, ETL/ELT pipelines, and data warehousing. Experience with serverless architectures, microservices, and event-driven design patterns. Strong technical skills with Python, Terraform and modern DevOps practices. Experience designing high-throughput, low-latency API solutions. Demonstrated technical leadership and mentoring abilities. Clear communication skills, with the ability to translate complex technical concepts. Strategic thinker, love white-boarding, and keen on mentoring engineers. Desirable: Experience with AI and machine learning architecture patterns. AWS Solution Architect – Pro certification. Show more Show less
Posted 1 week ago
3.0 - 5.0 years
0 Lacs
Pune, Maharashtra, India
On-site
The HiLabs Story HiLabs is a leading provider of AI-powered solutions to clean dirty data, unlocking its hidden potential for healthcare transformation. HiLabs is committed to transforming the healthcare industry through innovation, collaboration, and a relentless focus on improving patient outcomes. HiLabs Team Multidisciplinary industry leaders Healthcare domain experts AI/ML and data science experts Professionals hailing from the worlds best universities, business schools, and engineering institutes including Harvard, Yale, Carnegie Mellon, Duke, Georgia Tech, Indian Institute of Management (IIM), and Indian Institute of Technology (IIT). Be a part of a team that harnesses advanced AI, ML, and big data technologies to develop cutting-edge healthcare technology platform, delivering innovative business solutions. Job Title : Data Engineer I/II Job Location : Pune, Maharashtra, India Job summary: We are a leading Software as a Service (SaaS) company that specializes in the transformation of data in the US healthcare industry through cutting-edge Artificial Intelligence (AI) solutions. We are looking for Software Developers, who should continually strive to advance engineering excellence and technology innovation. The mission is to power the next generation of digital products and services through innovation, collaboration, and transparency. You will be a technology leader and doer who enjoys working in a dynamic, fast-paced environment. Responsibilities Design, develop, and maintain robust and scalable ETL/ELT pipelines to ingest and transform large datasets from various sources. Optimize and manage databases (SQL/NoSQL) to ensure efficient data storage, retrieval, and manipulation for both structured and unstructured data. Collaborate with data scientists, analysts, and engineers to integrate data from disparate sources and ensure smooth data flow between systems. Implement and maintain data validation and monitoring processes to ensure data accuracy, consistency, and availability. Automate repetitive data engineering tasks and optimize data workflows for performance and scalability. Work closely with cross-functional teams to understand their data needs and provide solutions that help scale operations. Ensure proper documentation of data engineering processes, workflows, and infrastructure for easy maintenance and scalability Desired Profile Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field. 3-5 years of hands-on experience as a Data Engineer or in a related data-driven role. Strong experience with ETL tools like Apache Airflow, Talend, or Informatica. Expertise in SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB, Cassandra). Strong proficiency in Python, Scala, or Java for data manipulation and pipeline development. Experience with cloud-based platforms (AWS, Google Cloud, Azure) and their data services (e.g., S3, Redshift, BigQuery). Familiarity with big data processing frameworks such as Hadoop, Spark, or Flink. Experience in data warehousing concepts and building data models (e.g., Snowflake, Redshift). Understanding of data governance, data security best practices, and data privacy regulations (e.g., GDPR, HIPAA). Familiarity with version control systems like Git.. HiLabs is an equal opportunity employer (EOE). No job applicant or employee shall receive less favorable treatment or be disadvantaged because of their gender, marital or family status, color, race, ethnic origin, religion, disability, or age; nor be subject to less favorable treatment or be disadvantaged on any other basis prohibited by applicable law. HiLabs is proud to be an equal opportunity workplace dedicated to pursuing and hiring a diverse and inclusive workforce to support individual growth and superior business results. Thank you for reviewing this opportunity with HiLabs! If this position appears to be a good fit for your skillset, we welcome your application. HiLabs Total Rewards Competitive Salary, Accelerated Incentive Policies, H1B sponsorship, Comprehensive benefits package that includes ESOPs, financial contribution for your ongoing professional and personal development, medical coverage for you and your loved ones, 401k, PTOs & a collaborative working environment, Smart mentorship, and highly qualified multidisciplinary, incredibly talented professionals from highly renowned and accredited medical schools, business schools, and engineering institutes. CCPA disclosure notice - https://www.hilabs.com/privacy Show more Show less
Posted 1 week ago
5.0 years
0 Lacs
Pune, Maharashtra, India
On-site
Job Summary: We are seeking an experienced Data Engineer with a strong background in Scala development, advanced SQL, and big data technologies, particularly Apache Spark. The candidate will be responsible for designing, building, optimizing, and maintaining highly scalable and reliable data pipelines and data infrastructure. Key Responsibilities: Data Pipeline Development: Design, develop, test, and deploy robust, high-performance, and scalable ETL/ELT data pipelines using Scala and Apache Spark to ingest, process, and transform large volumes of structured and unstructured data from diverse sources. Big Data Expertise: Leverage expertise in the Hadoop ecosystem (HDFS, Hive, etc.) and distributed computing principles to build efficient and fault-tolerant data solutions. Advanced SQL: Write complex, optimized SQL queries and stored procedures. Performance Optimization: Continuously monitor, analyze, and optimize the performance of data pipelines and data stores. Troubleshoot complex data-related issues, identify bottlenecks, and implement solutions for improved efficiency and reliability. Data Quality & Governance: Implement data quality checks, validation rules, and reconciliation processes to ensure the accuracy, completeness, and consistency of data. Contribute to data governance and security best practices. Automation & CI/CD: Implement automation for data pipeline deployment, monitoring, and alerting using tools like Apache Airflow, Jenkins, or similar CI/CD platforms. Documentation: Create and maintain comprehensive technical documentation for data architectures, pipelines, and processes. Required Skills & Qualifications: Bachelor's or master's degree in computer science, Engineering, or a related quantitative field. Minimum 5 years of professional experience in Data Engineering, with a strong focus on big data technologies. Proficiency in Scala for developing big data applications and transformations, especially with Apache Spark. Expert-level proficiency in SQL ; ability to write complex queries, optimize performance, and understand database internals. Extensive hands-on experience with Apache Spark (Spark SQL, Data Frames, RDDs) for large-scale data processing and analytics. Solid understanding of distributed computing concepts and experience with the Hadoop ecosystem (HDFS, Hive). Experience with building and optimizing ETL/ELT processes and data warehousing concepts. Show more Show less
Posted 1 week ago
8.0 years
0 Lacs
Gurgaon, Haryana, India
On-site
Role Overview We are seeking a highly skilled and forward-thinking professional to lead our Data Engineering and Data Science initiatives. As a Lead – DE + DS , you will play a critical role in designing and scaling data pipelines, architecting data platforms, and developing predictive models that drive strategic decision-making across the organization. This is a hybrid leadership role combining hands-on technical expertise with people management and stakeholder engagement. Key Responsibilities Data Engineering: Architect and manage scalable and secure data pipelines and ETL/ELT processes using cloud-based platforms (e.g., AWS, Azure, GCP) Design and maintain data lake/data warehouse structures and ensure data quality, availability, and governance Collaborate with DevOps and platform teams to automate data workflows and deploy pipelines in production Data Science Lead the development, deployment, and monitoring of machine learning models for business use cases (e.g., forecasting, recommendation engines, anomaly detection) Drive experimentation and advanced analytics using statistical, machine learning, and deep learning methods Translate business problems into data-driven solutions and actionable insights Leadership & Collaboration Lead and mentor a team of data engineers and data scientists, fostering skill development and collaboration Partner with business stakeholders, product owners, and engineering teams to align on data strategies and deliver impactful outcomes Define and enforce best practices in data architecture, coding standards, and model lifecycle management Required Skills & Qualifications Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, Mathematics, or a related field 8+ years of relevant experience in data engineering and/or data science, with at least 2 years in a technical leadership role Proficiency in SQL, Python, Spark, and distributed data processing frameworks Experience with data warehousing (Snowflake, Redshift, BigQuery), and data pipeline tools (Airflow, dbt, etc.) Strong understanding of ML frameworks (Scikit-learn, TensorFlow, PyTorch) and model deployment practices Solid grasp of data governance, MLOps, and CI/CD practices in a cloud environment Excellent communication and stakeholder management skills Preferred Qualifications Experience in Agile delivery environments Certifications in cloud platforms (e.g., AWS Certified Data Analytics, GCP Professional Data Engineer) Exposure to real-time data streaming (Kafka, Kinesis, etc.) Familiarity with visualization tools like Power BI, Tableau, or Looker Show more Show less
Posted 1 week ago
5.0 years
0 Lacs
Pune, Maharashtra, India
On-site
Description About Tripstack - We are travel tech entrepreneurs, changing the way millions of people travel. Our proprietary virtual interlining technology provides access to billions of travel itineraries by combining flights from different airline carriers that don’t traditionally work together. We take our customers from point A to B via C, at the lowest possible price. We are impacting the way people travel and provide higher margin opportunities to our partners that are some of the largest online travel agencies in the world. We pride ourselves on the performance-driven environment we have created for our teams to prosper and excel in. We come to work ready, to challenge and be challenged. We’re big enough to give our teams support but small enough that every person makes a difference. There are still plenty of challenges to champion. Requirements The Role - We are seeking an experienced data engineer to join our Data Engineering team embedded within the Data organization at TripStack. Responsibilities - Analyze and Organize Raw Data: Collect, clean, and structure raw data from diverse sources to make it suitable for further analysis and processing. Develop Robust Data Systems and Pipelines: Design and implement resilient data systems and pipelines to support efficient data processing, storage, and retrieval and establish data contracts with engineering teams. Ensure Data Meets Business Needs: Ensure that datasets are properly prepared and maintained to meet the reporting and analytics needs of the business. Prepare Data for Machine Learning: Collaborate with data scientists to prepare and process data for machine learning initiatives, ensuring compatibility and readiness for model training. Enhance Data Quality and Reliability: Implement processes and technologies to improve data quality and reliability, including data validation, cleansing, and deduplication. Collaborate on Analytics Data Flow Design: Work closely with data scientists and data architects to design and optimize the flow of analytics data, ensuring seamless integration and efficient data usage across the organization. Requirements gathering: Collaborate with cross-functional teams to gather and document business requirements for large-scale data engineering projects, ensuring clear understanding of stakeholder needs. Desired Skills & Experience - Bachelor’s degree in Computer Science or equivalent 5+ years of experience in data engineering or similar roles Proficiency in Python Experience working with structured and unstructured data Experience with big data technologies eg. Spark and Kafka and Apache Druid Strong data modeling and SQL skills Experience with orchestration tools eg. Apache Airflow, DBT, Databricks Strong cross-functional collaboration skills Nice to have - Master’s Degree in computer science, mathematics, engineering, or related discipline with 3+ years of experience Experience with MLOPs tools eg. MLFlow Airline travel industry experience is a plus Benefits What it takes to succeed here : Ambition and dedication to make a difference and change the way people travel; Where we always play to each other strength in a high performing team reaching for our common goal. We hold ourselves to the highest expectations, and move with a sense of urgency and hold ourselves accountable and win by staying true to what we believe in. What we offer : We offer an opportunity to work with a young, dynamic, and a growing team composed of high-caliber professionals. We value professionalism and promote a culture where individuals are encouraged to do more and be more. If you feel you share our passion for excellence, and growth, then look no further. We have an ambitious mission, and we need a world-class team to make it a reality. Upgrade to a First Class team! Show more Show less
Posted 1 week ago
5.0 years
0 Lacs
Mumbai Metropolitan Region
On-site
Relocation Assistance Offered Within Country Job Number #163961 - Mumbai, Maharashtra, India Who We Are Colgate-Palmolive Company is a global consumer products company operating in over 200 countries specializing in Oral Care, Personal Care, Home Care, Skin Care, and Pet Nutrition. Our products are trusted in more households than any other brand in the world, making us a household name! Join Colgate-Palmolive, a caring, innovative growth company reimagining a healthier future for people, their pets, and our planet. Guided by our core values—Caring, Inclusive, and Courageous—we foster a culture that inspires our people to achieve common goals. Together, let's build a brighter, healthier future for all. About Colgate-Palmolive Do you want to come to work with a smile and leave with one as well? In between those smiles, your day consists of working in a global organization, continually learning and collaborating, having stimulating discussions, and making impactful contributions! If this is how you see your career, Colgate is the place to be! Our diligent household brands, dedicated employees, and sustainability commitments make us a company passionate about building a future to smile about for our employees, consumers, and surrounding communities. The pride in our brand fuels a workplace that encourages creative thinking, champions experimentation, and promotes authenticity which has contributed to our enduring success. If you want to work for a company that lives by their values, then give your career a reason to smile...every single day. The Experience In today’s dynamic analytical / technological environment, it is an exciting time to be a part of the GLOBAL ANALYTICS team at Colgate. Our highly insight driven and innovative team is dedicated to driving growth for Colgate Palmolive in this constantly evolving landscape. What role will you play as a member of Colgate's Analytics team? The GLOBAL DATA SCIENCE & ADVANCED ANALYTICS vertical in Colgate Palmolive is focused on working on cases which have big $ impact and scope for scalability. With clear focus on addressing the business questions, with recommended actions The Data Scientist position would lead GLOBAL DATA SCIENCE & ADVANCED ANALYTICS projects within the Analytics Continuum. Conceptualizes and builds predictive modelling, simulations, and optimization solutions for clear $ objectives and measured value The Data Scientist would work on a range of projects ranging across Revenue Growth Management, Market Effectiveness, Forecasting etc. Data Scientist needs to handle relationships independently with Business and to drive projects such as Price Promotion, Marketing Mix and Forecasting Who are you…? You are a function expert - Leads GLOBAL DATA SCIENCE & ADVANCED ANALYTICS within the Analytics Continuum Conceptualizes and builds predictive modelling, simulations, and optimization solutions to address business questions or use cases Applies ML and AI to analytics algorithms to build inferential and predictive models allowing for scalable solutions to be deployed across the business Conducts model validations and continuous improvement of the algorithms, capabilities, or solutions built Deploys models using Airflow, Docker on Google Cloud Platforms Develops end to end business solutions from data extraction, data preparation, data mining to statistical modeling and then building business presentations Own Pricing and Promotion, Marketing Mix, Forecasting study from scoping to delivery Study large amounts of data to discover trends and patterns Mine data through various technologies like BigQuery and SQL Present insights in an easy to interpret way to the business teams Develop visualization (e.g. Looker, PyDash, Flask, PlotLy) using large datasets Ready to work closely with business partners across geographies You connect the dots - Merge multiple data sources and build Statistical Models / Machine Learning models in Price and Promo Elasticity Modelling, Marketing Mix Modelling to derive actionable business insights and recommendation Assemble large, sophisticated data sets that meet functional / non-functional business requirements Build data and visualization tools for Business analytics to assist them in decision making You are a collaborator - Work closely with Division Analytics team leads Work with data and analytics specialists across functions to drive data solutions You are an innovator - Identify, design, and implement new algorithms, process improvements: while continuously automating processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Qualifications What you’ll need BE/BTECH [ Computer Science, Information Technology is preferred], MBA or PGDM in Business Analytics / Data Science, Additional DS Certifications or Courses, MSC / MSTAT in Economics or Statistics 5+ years of experience in building data models and driving insights Hands-on/experience on developing statistical models, such as linear regression, ridge regression, lasso, random forest, SVM, gradient boosting, logistic regression, K-Means Clustering, Hierarchical Clustering, Bayesian Regression etc. Hands on experience on coding languages Python(mandatory), R, SQL, PySpark, SparkR Good Understanding of Cloud Frameworks Google Cloud, Snowflake and services like Kubernetes, Cloud Build, Cloud Run. Knowledge of using GitHub, Airflow for coding and model executions and model deployment on cloud platforms Solid understanding on tools like Looker, Domo, Power BI and web apps framework using plotly, pydash, sql Experience front facing Business teams (Client facing role) supporting and working with multi-functional teams in a dynamic environment What You’ll Need…(Preferred) Handling, redefining, and developing statistical models for RGM/Pricing and/or Marketing Effectiveness Experience with third-party data i.e., syndicated market data, Point of Sales, etc. Working knowledge of consumer-packaged goods industry Knowledge of machine learning techniques (clustering, decision tree learning, artificial neural networks, etc.) and their real-world advantages/drawbacks. Experience visualizing/communicating data for partners using: Tableau, DOMO, pydash, plotly, d3.js, ggplot2, pydash, R Shiny etc Willingness and ability to experiment with new tools and techniques Ability to maintain personal composure and thoughtfully handle difficult situations. Knowledge of Google products (Big Query, data studio, colab, Google Slides, Google Sheets etc) Knowledge of deployment of models in Cloud Environment using Airflow, Docker Ability to work with cross functional teams in IT, Data Architecture to build enterprise level Data Science products. Our Commitment to Diversity, Equity & Inclusion Achieving our purpose starts with our people — ensuring our workforce represents the people and communities we serve —and creating an environment where our people feel they belong; where we can be our authentic selves, feel treated with respect and have the support of leadership to impact the business in a meaningful way. Equal Opportunity Employer Colgate is an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity, sexual orientation, national origin, ethnicity, age, disability, marital status, veteran status (United States positions), or any other characteristic protected by law. Reasonable accommodation during the application process is available for persons with disabilities. Please complete this request form should you require accommodation. Show more Show less
Posted 1 week ago
6.0 - 9.0 years
0 Lacs
Gurugram, Haryana, India
On-site
Summary Position Summary Strategy & Analytics AI & Data In this age of disruption, organizations need to navigate the future with confidence, embracing decision making with clear, data-driven choices that deliver enterprise value in a dynamic business environment. The AI & Data team leverages the power of data, analytics, robotics, science and cognitive technologies to uncover hidden relationships from vast troves of data, generate insights, and inform decision-making. Together with the Strategy practice, our Strategy & Analytics portfolio helps clients transform their business by architecting organizational intelligence programs and differentiated strategies to win in their chosen markets. AI & Data will work with our clients to: Implement large-scale data ecosystems including data management, governance and the integration of structured and unstructured data to generate insights leveraging cloud-based platforms Leverage automation, cognitive and science-based techniques to manage data, predict scenarios and prescribe actions Drive operational efficiency by maintaining their data ecosystems, sourcing analytics expertise and providing As-a-Service offerings for continuous insights and improvements PySpark Sr. Consultant The position is suited for individuals who have demonstrated ability to work effectively in a fast paced, high volume, deadline driven environment. Education And Experience Education: B.Tech/M.Tech/MCA/MS 6-9 years of experience in design and implementation of migrating an Enterprise legacy system to Big Data Ecosystem for Data Warehousing project. Required Skills Must have excellent knowledge in Apache Spark and Python programming experience Deep technical understanding of distributed computing and broader awareness of different Spark version Strong UNIX operating system concepts and shell scripting knowledge Hands-on experience using Spark & Python Deep experience in developing data processing tasks using PySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations. Experience in deployment and operationalizing the code, knowledge of scheduling tools like Airflow, Control-M etc. is preferred Working experience on AWS ecosystem, Google Cloud, BigQuery etc. is an added advantage Hands on experience with AWS S3 Filesystem operations Good knowledge of Hadoop, Hive and Cloudera/ Hortonworks Data Platform Should have exposure with Jenkins or equivalent CICD tool & Git repository Experience handling CDC operations for huge volume of data Should understand and have operating experience with Agile delivery model Should have experience in Spark related performance tuning Should be well versed with understanding of design documents like HLD, TDD etc Should be well versed with Data historical load and overall Framework concepts Should have participated in different kinds of testing like Unit Testing, System Testing, User Acceptance Testing, etc Preferred Skills Exposure to PySpark, Cloudera/ Hortonworks, Hadoop and Hive. Exposure to AWS S3/EC2 and Apache Airflow Participation in client interactions/meetings is desirable. Participation in code-tuning is desirable. Recruiting tips From developing a stand out resume to putting your best foot forward in the interview, we want you to feel prepared and confident as you explore opportunities at Deloitte. Check out recruiting tips from Deloitte recruiters. Benefits At Deloitte, we know that great people make a great organization. We value our people and offer employees a broad range of benefits. Learn more about what working at Deloitte can mean for you. Our people and culture Our inclusive culture empowers our people to be who they are, contribute their unique perspectives, and make a difference individually and collectively. It enables us to leverage different ideas and perspectives, and bring more creativity and innovation to help solve our clients' most complex challenges. This makes Deloitte one of the most rewarding places to work. Our purpose Deloitte’s purpose is to make an impact that matters for our people, clients, and communities. At Deloitte, purpose is synonymous with how we work every day. It defines who we are. Our purpose comes through in our work with clients that enables impact and value in their organizations, as well as through our own investments, commitments, and actions across areas that help drive positive outcomes for our communities. Professional development From entry-level employees to senior leaders, we believe there’s always room to learn. We offer opportunities to build new skills, take on leadership opportunities and connect and grow through mentorship. From on-the-job learning experiences to formal development programs, our professionals have a variety of opportunities to continue to grow throughout their career. Requisition code: 300041 Show more Show less
Posted 1 week ago
3.0 - 6.0 years
0 Lacs
Gurugram, Haryana, India
On-site
Summary Position Summary Strategy & Analytics AI & Data In this age of disruption, organizations need to navigate the future with confidence, embracing decision making with clear, data-driven choices that deliver enterprise value in a dynamic business environment. The AI & Data team leverages the power of data, analytics, robotics, science and cognitive technologies to uncover hidden relationships from vast troves of data, generate insights, and inform decision-making. Together with the Strategy practice, our Strategy & Analytics portfolio helps clients transform their business by architecting organizational intelligence programs and differentiated strategies to win in their chosen markets. AI & Data will work with our clients to: Implement large-scale data ecosystems including data management, governance and the integration of structured and unstructured data to generate insights leveraging cloud-based platforms Leverage automation, cognitive and science-based techniques to manage data, predict scenarios and prescribe actions Drive operational efficiency by maintaining their data ecosystems, sourcing analytics expertise and providing As-a-Service offerings for continuous insights and improvements PySpark Consultant The position is suited for individuals who have demonstrated ability to work effectively in a fast paced, high volume, deadline driven environment. Education And Experience Education: B.Tech/M.Tech/MCA/MS 3-6 years of experience in design and implementation of migrating an Enterprise legacy system to Big Data Ecosystem for Data Warehousing project. Required Skills Must have excellent knowledge in Apache Spark and Python programming experience Deep technical understanding of distributed computing and broader awareness of different Spark version Strong UNIX operating system concepts and shell scripting knowledge Hands-on experience using Spark & Python Deep experience in developing data processing tasks using PySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations. Experience in deployment and operationalizing the code, knowledge of scheduling tools like Airflow, Control-M etc. is preferred Working experience on AWS ecosystem, Google Cloud, BigQuery etc. is an added advantage Hands on experience with AWS S3 Filesystem operations Good knowledge of Hadoop, Hive and Cloudera/ Hortonworks Data Platform Should have exposure with Jenkins or equivalent CICD tool & Git repository Experience handling CDC operations for huge volume of data Should understand and have operating experience with Agile delivery model Should have experience in Spark related performance tuning Should be well versed with understanding of design documents like HLD, TDD etc Should be well versed with Data historical load and overall Framework concepts Should have participated in different kinds of testing like Unit Testing, System Testing, User Acceptance Testing, etc Preferred Skills Exposure to PySpark, Cloudera/ Hortonworks, Hadoop and Hive. Exposure to AWS S3/EC2 and Apache Airflow Participation in client interactions/meetings is desirable. Participation in code-tuning is desirable. Recruiting tips From developing a stand out resume to putting your best foot forward in the interview, we want you to feel prepared and confident as you explore opportunities at Deloitte. Check out recruiting tips from Deloitte recruiters. Benefits At Deloitte, we know that great people make a great organization. We value our people and offer employees a broad range of benefits. Learn more about what working at Deloitte can mean for you. Our people and culture Our inclusive culture empowers our people to be who they are, contribute their unique perspectives, and make a difference individually and collectively. It enables us to leverage different ideas and perspectives, and bring more creativity and innovation to help solve our clients' most complex challenges. This makes Deloitte one of the most rewarding places to work. Our purpose Deloitte’s purpose is to make an impact that matters for our people, clients, and communities. At Deloitte, purpose is synonymous with how we work every day. It defines who we are. Our purpose comes through in our work with clients that enables impact and value in their organizations, as well as through our own investments, commitments, and actions across areas that help drive positive outcomes for our communities. Professional development From entry-level employees to senior leaders, we believe there’s always room to learn. We offer opportunities to build new skills, take on leadership opportunities and connect and grow through mentorship. From on-the-job learning experiences to formal development programs, our professionals have a variety of opportunities to continue to grow throughout their career. Requisition code: 300028 Show more Show less
Posted 1 week ago
5.0 - 10.0 years
0 Lacs
Gurugram, Haryana, India
On-site
Job Responsibilities Manage and maintain Azure Kubernetes Services. Establish, deploy, and maintain CI/CD pipelines to automate the build, test, and deployment processes. Investigate and resolve issues related to the application infrastructure, continuous integration, and deployment pipelines. Desired Skills And Experience Candidate Profile ◼ 5 to 10 years of experience in a DevOps role preferably in Investment Banking. ◼ Certified Kubernetes Administrator having certification status in the active state. ◼ Experience in managing and working with Kubernetes environments and observability tools. ◼ Strong knowledge of containerization and orchestration of microservices. ◼ Experience with Docker/Podman, Helm, ArgoCD GitOps tool, Terraform. ◼ Experience with Azure Infrastructure including Entra ID, Azure Kubernetes Service, Azure Storage, Azure Redis,and other Azure cloud related technologies. ◼ Experience with Prometheus, Grafana, Loki, Tempo, Grafana Agent, Azure Monitor logging and observability tools. ◼ Good exposure to Bamboo CI/CD tools, Bitbucket, GIT. ◼ Experience with production environment troubleshooting and debugging. ◼ Automation scripting (Bash, Powershell, Python). ◼ Good exposure on git branching strategies. ◼ Be able to demonstrate a high level of professionalism, organisation, self-motivation, and a desire for self improvement. ◼ Should be self-driven and proactive having ability to plan, schedule and manage a demanding workload. Nice To Have Skills ◼ Implement backup and disaster recovery strategies and participate in annual DR tests and assist with executing the DR test plan. ◼ Develop and utilize cost tracking tools and methodologies to provide transparent and accurate financial reporting for all projects. Identify areas where cloud spend can be optimized to reduce wastage and costs. ◼ Good knowledge of scheduling jobs via Apache Airflow. ◼ Good knowledge of Azure Landing zone, Azure networking concepts such as private links. ◼ Good knowledge or experience in deploying and maintaining Azure Databricks infra. ◼ Good Java, NodeJs skills. ◼ Good understanding of Kafka streaming and MongoDB. ◼ Knowledge of DevSecOps practices. Key Responsibilities ◼ Implement and maintain infrastructure-as-code (IaC) using tools such as Terraform. ◼ Utilize containerization technologies like Azure Kubernetes to orchestrate and manage containerized applications in a production environment. ◼ Manage and maintain the lifecycle of core application suite that provide common capabilities such as continuous deployment, observability, and Kafka streaming. ◼ Monitor and troubleshoot infrastructure and application issues using monitoring tools. ◼ Collaborate with infra teams to provision and manage infra resources required by FO IT development teams in Azure cloud. ◼ Establish, deploy, and maintain CI/CD pipelines to automate the build, test, and deployment processes. ◼ Investigate and resolve issues related to the application infrastructure, continuous integration, and deployment pipelines. ◼ Identify areas that benefit from automation and build automated processes wherever possible. ◼ Design and develop application health dashboards, alerting and notification delivery systems to help with observability of application stack in Azure cloud. ◼ Collaborate with development, testing, and operations teams to gather, understand, and analyze functional requirements. ◼ Implement and enforce security best practices throughout the infrastructure, including identity and access management (RBAC), encryption, and secure network configurations. Show more Show less
Posted 1 week ago
6.0 - 9.0 years
0 Lacs
Greater Kolkata Area
On-site
Summary Position Summary Strategy & Analytics AI & Data In this age of disruption, organizations need to navigate the future with confidence, embracing decision making with clear, data-driven choices that deliver enterprise value in a dynamic business environment. The AI & Data team leverages the power of data, analytics, robotics, science and cognitive technologies to uncover hidden relationships from vast troves of data, generate insights, and inform decision-making. Together with the Strategy practice, our Strategy & Analytics portfolio helps clients transform their business by architecting organizational intelligence programs and differentiated strategies to win in their chosen markets. AI & Data will work with our clients to: Implement large-scale data ecosystems including data management, governance and the integration of structured and unstructured data to generate insights leveraging cloud-based platforms Leverage automation, cognitive and science-based techniques to manage data, predict scenarios and prescribe actions Drive operational efficiency by maintaining their data ecosystems, sourcing analytics expertise and providing As-a-Service offerings for continuous insights and improvements PySpark Sr. Consultant The position is suited for individuals who have demonstrated ability to work effectively in a fast paced, high volume, deadline driven environment. Education And Experience Education: B.Tech/M.Tech/MCA/MS 6-9 years of experience in design and implementation of migrating an Enterprise legacy system to Big Data Ecosystem for Data Warehousing project. Required Skills Must have excellent knowledge in Apache Spark and Python programming experience Deep technical understanding of distributed computing and broader awareness of different Spark version Strong UNIX operating system concepts and shell scripting knowledge Hands-on experience using Spark & Python Deep experience in developing data processing tasks using PySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations. Experience in deployment and operationalizing the code, knowledge of scheduling tools like Airflow, Control-M etc. is preferred Working experience on AWS ecosystem, Google Cloud, BigQuery etc. is an added advantage Hands on experience with AWS S3 Filesystem operations Good knowledge of Hadoop, Hive and Cloudera/ Hortonworks Data Platform Should have exposure with Jenkins or equivalent CICD tool & Git repository Experience handling CDC operations for huge volume of data Should understand and have operating experience with Agile delivery model Should have experience in Spark related performance tuning Should be well versed with understanding of design documents like HLD, TDD etc Should be well versed with Data historical load and overall Framework concepts Should have participated in different kinds of testing like Unit Testing, System Testing, User Acceptance Testing, etc Preferred Skills Exposure to PySpark, Cloudera/ Hortonworks, Hadoop and Hive. Exposure to AWS S3/EC2 and Apache Airflow Participation in client interactions/meetings is desirable. Participation in code-tuning is desirable. Recruiting tips From developing a stand out resume to putting your best foot forward in the interview, we want you to feel prepared and confident as you explore opportunities at Deloitte. Check out recruiting tips from Deloitte recruiters. Benefits At Deloitte, we know that great people make a great organization. We value our people and offer employees a broad range of benefits. Learn more about what working at Deloitte can mean for you. Our people and culture Our inclusive culture empowers our people to be who they are, contribute their unique perspectives, and make a difference individually and collectively. It enables us to leverage different ideas and perspectives, and bring more creativity and innovation to help solve our clients' most complex challenges. This makes Deloitte one of the most rewarding places to work. Our purpose Deloitte’s purpose is to make an impact that matters for our people, clients, and communities. At Deloitte, purpose is synonymous with how we work every day. It defines who we are. Our purpose comes through in our work with clients that enables impact and value in their organizations, as well as through our own investments, commitments, and actions across areas that help drive positive outcomes for our communities. Professional development From entry-level employees to senior leaders, we believe there’s always room to learn. We offer opportunities to build new skills, take on leadership opportunities and connect and grow through mentorship. From on-the-job learning experiences to formal development programs, our professionals have a variety of opportunities to continue to grow throughout their career. Requisition code: 300041 Show more Show less
Posted 1 week ago
3.0 - 6.0 years
0 Lacs
Greater Kolkata Area
On-site
Summary Position Summary Strategy & Analytics AI & Data In this age of disruption, organizations need to navigate the future with confidence, embracing decision making with clear, data-driven choices that deliver enterprise value in a dynamic business environment. The AI & Data team leverages the power of data, analytics, robotics, science and cognitive technologies to uncover hidden relationships from vast troves of data, generate insights, and inform decision-making. Together with the Strategy practice, our Strategy & Analytics portfolio helps clients transform their business by architecting organizational intelligence programs and differentiated strategies to win in their chosen markets. AI & Data will work with our clients to: Implement large-scale data ecosystems including data management, governance and the integration of structured and unstructured data to generate insights leveraging cloud-based platforms Leverage automation, cognitive and science-based techniques to manage data, predict scenarios and prescribe actions Drive operational efficiency by maintaining their data ecosystems, sourcing analytics expertise and providing As-a-Service offerings for continuous insights and improvements PySpark Consultant The position is suited for individuals who have demonstrated ability to work effectively in a fast paced, high volume, deadline driven environment. Education And Experience Education: B.Tech/M.Tech/MCA/MS 3-6 years of experience in design and implementation of migrating an Enterprise legacy system to Big Data Ecosystem for Data Warehousing project. Required Skills Must have excellent knowledge in Apache Spark and Python programming experience Deep technical understanding of distributed computing and broader awareness of different Spark version Strong UNIX operating system concepts and shell scripting knowledge Hands-on experience using Spark & Python Deep experience in developing data processing tasks using PySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations. Experience in deployment and operationalizing the code, knowledge of scheduling tools like Airflow, Control-M etc. is preferred Working experience on AWS ecosystem, Google Cloud, BigQuery etc. is an added advantage Hands on experience with AWS S3 Filesystem operations Good knowledge of Hadoop, Hive and Cloudera/ Hortonworks Data Platform Should have exposure with Jenkins or equivalent CICD tool & Git repository Experience handling CDC operations for huge volume of data Should understand and have operating experience with Agile delivery model Should have experience in Spark related performance tuning Should be well versed with understanding of design documents like HLD, TDD etc Should be well versed with Data historical load and overall Framework concepts Should have participated in different kinds of testing like Unit Testing, System Testing, User Acceptance Testing, etc Preferred Skills Exposure to PySpark, Cloudera/ Hortonworks, Hadoop and Hive. Exposure to AWS S3/EC2 and Apache Airflow Participation in client interactions/meetings is desirable. Participation in code-tuning is desirable. Recruiting tips From developing a stand out resume to putting your best foot forward in the interview, we want you to feel prepared and confident as you explore opportunities at Deloitte. Check out recruiting tips from Deloitte recruiters. Benefits At Deloitte, we know that great people make a great organization. We value our people and offer employees a broad range of benefits. Learn more about what working at Deloitte can mean for you. Our people and culture Our inclusive culture empowers our people to be who they are, contribute their unique perspectives, and make a difference individually and collectively. It enables us to leverage different ideas and perspectives, and bring more creativity and innovation to help solve our clients' most complex challenges. This makes Deloitte one of the most rewarding places to work. Our purpose Deloitte’s purpose is to make an impact that matters for our people, clients, and communities. At Deloitte, purpose is synonymous with how we work every day. It defines who we are. Our purpose comes through in our work with clients that enables impact and value in their organizations, as well as through our own investments, commitments, and actions across areas that help drive positive outcomes for our communities. Professional development From entry-level employees to senior leaders, we believe there’s always room to learn. We offer opportunities to build new skills, take on leadership opportunities and connect and grow through mentorship. From on-the-job learning experiences to formal development programs, our professionals have a variety of opportunities to continue to grow throughout their career. Requisition code: 300028 Show more Show less
Posted 1 week ago
5.0 - 10.0 years
15 - 30 Lacs
Vadodara
Remote
We are seeking an experienced Senior Data Engineer to join our team. The ideal candidate will have a strong background in data engineering and AWS infrastructure, with hands-on experience in building and maintaining data pipelines and the necessary infrastructure components. The role will involve using a mix of data engineering tools and AWS services to design, build, and optimize data architecture. Key Responsibilities: Design, develop, and maintain data pipelines using Airflow and AWS services. Implement and manage data warehousing solutions with Databricks and PostgreSQL. Automate tasks using GIT / Jenkins. Develop and optimize ETL processes, leveraging AWS services like S3, Lambda, AppFlow, and DMS. Create and maintain visual dashboards and reports using Looker. Collaborate with cross-functional teams to ensure smooth integration of infrastructure components. Ensure the scalability, reliability, and performance of data platforms. Work with Jenkins for infrastructure automation. Technical and functional areas of expertise: Working as a senior individual contributor on a data intensive project Strong experience in building high performance, resilient & secure data processing pipelines preferably using Python based stack. Extensive experience in building data intensive applications with a deep understanding of querying and modeling with relational databases preferably on time-series data. Intermediate proficiency in AWS services (S3, Airflow) Proficiency in Python and PySpark Proficiency with ThoughtSpot or Databricks. Intermediate proficiency in database scripting (SQL) Basic experience with Jenkins for task automation Nice to Have : Intermediate proficiency in data analytics tools (Power BI / Tableau / Looker / ThoughSpot) Experience working with AWS Lambda, Glue, AppFlow, and other AWS transfer services. Exposure to PySpark and data automation tools like Jenkins or CircleCI. Familiarity with Terraform for infrastructure-as-code. Experience in data quality testing to ensure the accuracy and reliability of data pipelines. Proven experience working directly with U.S. client stakeholders. Ability to work independently and take the lead on tasks. Education and experience: Bachelors or masters in computer science or related fields. 5+ years of experience Stack/Skills needed: Databricks PostgreSQL Python & Pyspark AWS Stack Power BI / Tableau / Looker / ThoughSpot Familiarity with GIT and/or CI/CD tools
Posted 1 week ago
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
The airflow job market in India is rapidly growing as more companies are adopting data pipelines and workflow automation. Airflow, an open-source platform, is widely used for orchestrating complex computational workflows and data processing pipelines. Job seekers with expertise in airflow can find lucrative opportunities in various industries such as technology, e-commerce, finance, and more.
The average salary range for airflow professionals in India varies based on experience levels: - Entry-level: INR 6-8 lakhs per annum - Mid-level: INR 10-15 lakhs per annum - Experienced: INR 18-25 lakhs per annum
In the field of airflow, a typical career path may progress as follows: - Junior Airflow Developer - Airflow Developer - Senior Airflow Developer - Airflow Tech Lead
In addition to airflow expertise, professionals in this field are often expected to have or develop skills in: - Python programming - ETL concepts - Database management (SQL) - Cloud platforms (AWS, GCP) - Data warehousing
As you explore job opportunities in the airflow domain in India, remember to showcase your expertise, skills, and experience confidently during interviews. Prepare well, stay updated with the latest trends in airflow, and demonstrate your problem-solving abilities to stand out in the competitive job market. Good luck!
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.