Very good experience on Continuous Flow Graph tool used for point based development. Design, develop, and maintain ETL processes using Ab Initio tools. Write, test, and deploy Ab Initio graphs, scripts, and other necessary components. Troubleshoot and resolve data processing issues and improve performance Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Over all 8 Years and Relevant 5+ years Extract, transform, and load data from various sources into data warehouses, operational data stores, or other target systems. Work with different data formats, including structured, semi-structured, and unstructured data Preferred technical and professional experience Effective communication and presentation skills. Industry expertise / specialization

Skilled Multiple GCP services - GCS, BigQuery, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer etc. Must have Python and SQL work experience & Proactive, collaborative and ability to respond to critical situation Ability to analyse data for functional business requirements & front face customer Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise 5 to 7 years of relevant experience working as technical analyst with Big Query on GCP platform. Skilled in multiple GCP services - GCS, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer You love collaborative environments that use agile methodologies to encourage creative design thinking and find innovative ways to develop with cutting edge technologies Ambitious individual who can work under their own direction towards agreed targets/goals and with creative approach to work Preferred technical and professional experience Create up to 3 bullets maxitive individual with an ability to manage change and proven time management Proven interpersonal skills while contributing to team effort by accomplishing related results as needed Up-to-date technical knowledge by attending educational workshops, reviewing publications (encouraging then to focus on required skills)

Skilled Multiple GCP services - GCS, BigQuery, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer etc. Must have Python and SQL work experience & Proactive, collaborative and ability to respond to critical situation Ability to analyse data for functional business requirements & front face customer Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise 5 to 7 years of relevant experience working as technical analyst with Big Query on GCP platform. Skilled in multiple GCP services - GCS, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer You love collaborative environments that use agile methodologies to encourage creative design thinking and find innovative ways to develop with cutting edge technologies Ambitious individual who can work under their own direction towards agreed targets/goals and with creative approach to work Preferred technical and professional experience Create up to 3 bullets maxitive individual with an ability to manage change and proven time management Proven interpersonal skills while contributing to team effort by accomplishing related results as needed Up-to-date technical knowledge by attending educational workshops, reviewing publications (encouraging then to focus on required skills)

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / Azure eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Total 5-8 years of experience in Data Management (DW, DL, Data Platform, Lakehouse) and Data Engineering skills Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala; Minimum 3 years of experience on Cloud Data Platforms on Azure; Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Good to excellent SQL skills Preferred technical and professional experience Certification in Azure and Data Bricks or Cloudera Spark Certified developers Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Knowledge or experience of Snowflake will be an added advantage

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / Azure eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Total 5-8 years of experience in Data Management (DW, DL, Data Platform, Lakehouse) and Data Engineering skills Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala; Minimum 3 years of experience on Cloud Data Platforms on Azure; Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Good to excellent SQL skills Preferred technical and professional experience Certification in Azure and Data Bricks or Cloudera Spark Certified developers Experience in DataBricks / Azure HDInsight / Azure Data Factory, Synapse, SQL Server DB Knowledge or experience of Snowflake will be an added advantage

As an Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporating a variety of statistical and machine learning techniques Designing and implementing various enterprise search applications such as Elasticsearch and Splunk for client requirements Work in an Agile, collaborative environment, partnering with other scientists, engineers, consultants and database administrators of all backgrounds and disciplines to bring analytical rigor and statistical methods to the challenges of predicting behaviours. Build teams or writing programs to cleanse and integrate data in an efficient and reusable manner, developing predictive or prescriptive models, and evaluating modeling results Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise We are seeking a skilled Azure Data Engineer with 5+ years of experience Including 3+ years of hands-on experience with ADF/Databricks The ideal candidate Data bricks,Data Lake, Phyton programming skills. The candidate will also have experience for deploying to data bricks. Familiarity with Azure Data Factory Preferred technical and professional experience Good communication skills. 3+ years of experience with ADF/DB/DataLake. Ability to communicate results to technical and non-technical audiences

As an Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporating a variety of statistical and machine learning techniques Designing and implementing various enterprise search applications such as Elasticsearch and Splunk for client requirements Work in an Agile, collaborative environment, partnering with other scientists, engineers, consultants and database administrators of all backgrounds and disciplines to bring analytical rigor and statistical methods to the challenges of predicting behaviours. Build teams or writing programs to cleanse and integrate data in an efficient and reusable manner, developing predictive or prescriptive models, and evaluating modeling results Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise We are seeking a skilled Azure Data Engineer with 5+ years of experience Including 3+ years of hands-on experience with ADF/Databricks The ideal candidate Data bricks,Data Lake, Phyton programming skills. The candidate will also have experience for deploying to data bricks. Familiarity with Azure Data Factory Preferred technical and professional experience Good communication skills. 3+ years of experience with ADF/DB/DataLake. Ability to communicate results to technical and non-technical audiences

Work with broader team to build, analyze and improve the AI solutions. You will also work with our software developers in consuming different enterprise applications Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Resource should have 5-7 years of experience. Sound knowledge of Python and should know how to use the ML related services. Proficient in Python with focus on Data Analytics Packages. Strategy Analyse large, complex data sets and provide actionable insights to inform business decisions. Strategy Design and implementing data models that help in identifying patterns and trends. Collaboration Work with data engineers to optimize and maintain data pipelines. Perform quantitative analyses that translate data into actionable insights and provide analytical, data-driven decision-making. Identify and recommend process improvements to enhance the efficiency of the data platform. Develop and maintain data models, algorithms, and statistical models Preferred technical and professional experience Experience with conversation analytics. Experience with cloud technologies Experience with data exploration tools such as Tableu

Skilled Multiple GCP services - GCS, Big Query, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer etc. Must have Python and SQL work experience & Proactive, collaborative and ability to respond to critical situation Ability to analyse data for functional business requirements & front face customer Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise 5 to 7 years of relevant experience working as technical analyst with Big Query on GCP platform. Skilled in multiple GCP services - GCS, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer You love collaborative environments that use agile methodologies to encourage creative design thinking and find innovative ways to develop with cutting edge technologies Ambitious individual who can work under their own direction towards agreed targets/goals and with creative approach to work Preferred technical and professional experience Create up to 3 bullets MA Intuitive individual with an ability to manage change and proven time management Proven interpersonal skills while contributing to team effort by accomplishing related results as needed Up-to-date technical knowledge by attending educational workshops, reviewing publications (encouraging then to focus on required skills

Skilled Multiple GCP services - GCS, Big Query, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer etc. Must have Python and SQL work experience & Proactive, collaborative and ability to respond to critical situation Ability to analyse data for functional business requirements & front face customer Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise 5 to 7 years of relevant experience working as technical analyst with Big Query on GCP platform. Skilled in multiple GCP services - GCS, Cloud SQL, Dataflow, Pub/Sub, Cloud Run, Workflow, Composer, Error reporting, Log explorer You love collaborative environments that use agile methodologies to encourage creative design thinking and find innovative ways to develop with cutting edge technologies Ambitious individual who can work under their own direction towards agreed targets/goals and with creative approach to work Preferred technical and professional experience Create up to 3 bullets MA Intuitive individual with an ability to manage change and proven time management Proven interpersonal skills while contributing to team effort by accomplishing related results as needed Up-to-date technical knowledge by attending educational workshops, reviewing publications (encouraging then to focus on required skills

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes to migrate and deploy data across various systems. You will collaborate with cross-functional teams to understand their data needs and provide effective solutions, ensuring that the data infrastructure is robust and scalable to meet the demands of the organization. Roles & Responsibilities:- Expected to be an SME.- Collaborate and manage the team to perform.- Responsible for team decisions.- Engage with multiple teams and contribute on key decisions.- Provide solutions to problems for their immediate team and across multiple teams.- Mentor junior team members to enhance their skills and knowledge in data engineering.- Continuously evaluate and improve data processes to enhance efficiency and effectiveness. Professional & Technical Skills: - Must To Have Skills: Proficiency in Databricks Unified Data Analytics Platform.- Experience with data integration and ETL tools.- Strong understanding of data modeling and database design principles.- Familiarity with cloud platforms and services related to data storage and processing.- Knowledge of programming languages such as Python or Scala for data manipulation. Additional Information:- The candidate should have minimum 5 years of experience in Databricks Unified Data Analytics Platform.- This position is based at our Bengaluru office.- A 15 years full time education is required. Qualification 15 years full time education

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes to migrate and deploy data across various systems. You will collaborate with cross-functional teams to understand their data needs and provide effective solutions, ensuring that the data infrastructure is robust and scalable to meet the demands of the organization. Roles & Responsibilities:- Expected to be an SME.- Collaborate and manage the team to perform.- Responsible for team decisions.- Engage with multiple teams and contribute on key decisions.- Provide solutions to problems for their immediate team and across multiple teams.- Mentor junior team members to enhance their skills and knowledge in data engineering.- Continuously evaluate and improve data processes to enhance efficiency and effectiveness. Professional & Technical Skills: - Must To Have Skills: Proficiency in Databricks Unified Data Analytics Platform.- Experience with data integration and ETL tools.- Strong understanding of data modeling and database design principles.- Familiarity with cloud platforms and services related to data storage and processing.- Knowledge of programming languages such as Python or Scala for data manipulation. Additional Information:- The candidate should have minimum 5 years of experience in Databricks Unified Data Analytics Platform.- This position is based at our Bengaluru office.- A 15 years full time education is required. Qualification 15 years full time education

Job Title: Chief Technology Officer (CTO) Role Overview: We are seeking a visionary Chief Technology Officer to lead our technology function and drive the development of innovative AdTech solutions. In this leadership role, you will define and implement the company's technical strategy while overseeing engineering, data science, and product technology teams. Your focus will be on building scalable, high-performance platforms including RTB, DSP, and SSP systems. Key Responsibilities: Develop and execute a forward-looking technology roadmap aligned with business goals. Lead cross-functional teams in engineering and product development. Architect and manage real-time bidding systems, data infrastructure, and platform scalability. Drive innovation in AI/ML, big data, and real-time analytics. Ensure system reliability, security, DevOps, and data privacy best practices. Collaborate with leadership to deliver impactful tech-driven products. Represent the company in technical partnerships and industry events. Requirements: 10+ years in software engineering, with 5+ in a leadership role. Strong background in AdTech (RTB, DSP, SSP, OpenRTB). Expertise in AI/ML, cloud (AWS/GCP), and big data (Kafka, Spark, Hadoop). Proven experience in building scalable backend systems and leading high-performing teams. Bachelor’s or Master’s in Computer Science or Engineering; MBA/PhD is a plus. Show more Show less

Job Title: Principal Data Engineer (MTS4 / Principal Engineer) About the Role As a Principal Data Engineer, you will drive the strategy, architecture, and execution of large-scale data solutions across our function. This role involves tackling highly ambiguous, complex challenges where the business problem may not be fully defined at the outset. You will partner closely with cross-functional teams (Engineering, Product, Operations) to shape and deliver our data roadmap. Your work will have a profound impact on our functions' data capabilities, influencing multiple teams’ technical and product direction. You should bring deep expertise in designing and developing robust data pipelines and platforms, leveraging technologies such as Spark, Airflow, Kafka, and other emerging tools. You will set standards and best practices that raise the bar for engineering excellence across the organization. Key Responsibilities Architect & Define Scope Own end-to-end design of critical data pipelines and platforms in an environment characterized by high ambiguity. Translate loosely defined business objectives into a clear technical plan, breaking down complex problems into achievable milestones. Technology Leadership & Influence Provide thought leadership in data engineering, driving the adoption of Spark, Airflow, Kafka, and other relevant technologies (e.g., Hadoop, Flink, Kubernetes, Snowflake, etc.). Lead design reviews and champion best practices for coding, system architecture, data quality, and reliability. Influence senior stakeholders (Engineers, EMs, Product Managers) on technology decisions and roadmap priorities. Execution & Delivery Spearhead strategic, multi-team projects that advance the organization’s data infrastructure and capabilities. Deconstruct complex architectures into simpler components that can be executed by various teams in parallel. Drive operational excellence, owning escalations and ensuring high availability, scalability, and cost-effectiveness of our data solutions. Mentor and develop engineering talent, fostering a culture of collaboration and continuous learning. Impact & Technical Complexity Shape how the organization operates by introducing innovative data solutions and strategic technical direction. Solve endemic, highly complex data engineering problems with robust, scalable, and cost-optimized solutions. Continuously balance short-term business needs with long-term architectural vision. Process Improvement & Best Practices Set and enforce engineering standards that elevate quality and productivity across multiple teams. Lead by example in code reviews, automation, CI/CD practices, and documentation. Champion a culture of continuous improvement, driving adoption of new tools and methodologies to keep our data ecosystem cutting-edge. Qualifications Education & Experience : Bachelor’s or Master’s degree in Computer Science, Engineering, or related field (or equivalent experience). 5+ years of software/data engineering experience, with significant exposure to large-scale distributed systems. Technical Expertise : Demonstrated proficiency with Spark, Airflow, Kafka, and at least one major programming language (e.g., Python, Scala, Java). Experience with data ecosystem technologies such as Hadoop, Flink, Snowflake, Kubernetes, etc. Proven track record of architecting and delivering highly scalable data infrastructure solutions. Leadership & Communication : Ability to navigate and bring clarity in ambiguous situations. Strong cross-functional collaboration skills, influencing both technical and non-technical stakeholders. Experience coaching and mentoring senior engineers. Problem-Solving : History of tackling complex, ambiguous data challenges and delivering tangible results. Comfort making informed trade-offs between opportunity vs. architectural complexity. Show more Show less

Requirements Minimum of 2-3 years of FullStack Software Development experience in building large-scale mission-critical applications. Strong foundation in computer science, with strong competencies in data structures, algorithms, and software design optimized for building highly distributed and parallelized systems. Proficiency in one or more programming languages - Java and Python. Strong hands-on experience in MEAN, MERN, Core Java, J2EE technologies, Microservices, Spring, Hibernate, SQL, REST APIs. Experience in web development using one of the technologies, like Angular or React, etc. Experience with one or more of the following database technologies: SQL Server, Postgres, MySQL, and NoSQL such as HBase, MongoDB, and DynamoDB. Strong problem-solving skills to deep dive, brainstorm, and choose the best solution approach. Experience with AWS Services like EKS, ECS, S3 EC2 RDS, Redshift, and Github/Stash, CI/CD Pipelines, Maven, Jenkins, Security Tools, Kubernetes/VMs/Linux, Monitoring, Alerting, etc. Experience in Agile development is a big plus. Excellent presentation, collaboration, and communication skills required. Result-oriented and experienced in leading broad initiatives and teams. Knowledge of Big Data technologies like Hadoop and Hive, Spark, Kafka, etc. would be an added advantage. Bachelor's or Master's degree in mathematics, Computer Science. 1-4 years of experience as a FullStackEngineer. Proven analytic skills and designing scalable applications. This job was posted by Vivek Chhikara from Protium. Show more Show less

Job Title: Principal Data Engineer (MTS4 / Principal Engineer) About the Role As a Principal Data Engineer, you will drive the strategy, architecture, and execution of large-scale data solutions across our function. This role involves tackling highly ambiguous, complex challenges where the business problem may not be fully defined at the outset. You will partner closely with cross-functional teams (Engineering, Product, Operations) to shape and deliver our data roadmap. Your work will have a profound impact on our functions' data capabilities, influencing multiple teams’ technical and product direction. You should bring deep expertise in designing and developing robust data pipelines and platforms, leveraging technologies such as Spark, Airflow, Kafka, and other emerging tools. You will set standards and best practices that raise the bar for engineering excellence across the organization. Key Responsibilities Architect & Define Scope Own end-to-end design of critical data pipelines and platforms in an environment characterized by high ambiguity. Translate loosely defined business objectives into a clear technical plan, breaking down complex problems into achievable milestones. Technology Leadership & Influence Provide thought leadership in data engineering, driving the adoption of Spark, Airflow, Kafka, and other relevant technologies (e.g., Hadoop, Flink, Kubernetes, Snowflake, etc.). Lead design reviews and champion best practices for coding, system architecture, data quality, and reliability. Influence senior stakeholders (Engineers, EMs, Product Managers) on technology decisions and roadmap priorities. Execution & Delivery Spearhead strategic, multi-team projects that advance the organization’s data infrastructure and capabilities. Deconstruct complex architectures into simpler components that can be executed by various teams in parallel. Drive operational excellence, owning escalations and ensuring high availability, scalability, and cost-effectiveness of our data solutions. Mentor and develop engineering talent, fostering a culture of collaboration and continuous learning. Impact & Technical Complexity Shape how the organization operates by introducing innovative data solutions and strategic technical direction. Solve endemic, highly complex data engineering problems with robust, scalable, and cost-optimized solutions. Continuously balance short-term business needs with long-term architectural vision. Process Improvement & Best Practices Set and enforce engineering standards that elevate quality and productivity across multiple teams. Lead by example in code reviews, automation, CI/CD practices, and documentation. Champion a culture of continuous improvement, driving adoption of new tools and methodologies to keep our data ecosystem cutting-edge. Qualifications Education & Experience : Bachelor’s or Master’s degree in Computer Science, Engineering, or related field (or equivalent experience). 5+ years of software/data engineering experience, with significant exposure to large-scale distributed systems. Technical Expertise : Demonstrated proficiency with Spark, Airflow, Kafka, and at least one major programming language (e.g., Python, Scala, Java). Experience with data ecosystem technologies such as Hadoop, Flink, Snowflake, Kubernetes, etc. Proven track record of architecting and delivering highly scalable data infrastructure solutions. Leadership & Communication : Ability to navigate and bring clarity in ambiguous situations. Strong cross-functional collaboration skills, influencing both technical and non-technical stakeholders. Experience coaching and mentoring senior engineers. Problem-Solving : History of tackling complex, ambiguous data challenges and delivering tangible results. Comfort making informed trade-offs between opportunity vs. architectural complexity. Show more Show less

Job Title: Cloud DevOps Architect Location: Pune, India Experience: 10 - 15 Years Work Mode: Full-time, Office-based Company : Smartavya Analytica Private Limited Company Overview: Smartavya Analytica is a niche Data and AI company based in Mumbai, established in 2017. We specialize in data-driven innovation, transforming enterprise data into strategic insights. With expertise spanning over 25+ Data Modernization projects and handling large datasets up to 24 PB in a single implementation, we have successfully delivered data and AI projects across multiple industries, including retail, finance, telecom, manufacturing, insurance, and capital markets. We are specialists in Cloud, Hadoop, Big Data, AI, and Analytics, with a strong focus on Data Modernization for OnPremises, Private, and Public Cloud Platforms. Visit us at: https://smart-analytica.com Job Summary: We are looking for an accomplished Cloud DevOps Architect to design and implement robust DevOps and Infrastructure Automation frameworks across Azure, GCP, or AWS environments. The ideal candidate will have a deep understanding of CI/CD , IaC , VPC Networking , Security , and Automation using Terraform or Ansible . Key Responsibilities: Architect and build end-to-end DevOps pipelines using native cloud services (Azure DevOps, AWS CodePipeline, GCP Cloud Build) and third-party tools (Jenkins, GitLab, etc.). Define and implement foundation setup architecture (Azure, GCP and AWS) as per the recommended best practices. Design and deploy secure VPC architectures , manage networking, security groups, load balancers, and VPN gateways. Implement Infrastructure as Code (IaC) using Terraform or Ansible for scalable and repeatable deployments. Establish CI/CD frameworks integrating with Git, containers, and orchestration tools (e.g., Kubernetes, ECS, AKS, GKE). Define and enforce cloud security best practices including IAM, encryption, secrets management, and compliance standards. Collaborate with application, data, and security teams to optimize infrastructure, release cycles, and system performance. Drive continuous improvement in automation, observability, and incident response practices. Must-Have Skills: 10- 5 years of experience in DevOps, Infrastructure, or Cloud Architecture roles. Deep hands-on expertise in Azure , GCP , or AWS cloud platforms (any one is mandatory, more is a bonus). Strong knowledge of VPC architecture , Cloud Security , IAM , and Networking principles . Expertise in Terraform or Ansible for Infrastructure as Code. Experience building resilient CI/CD pipelines and automating application deployments. Strong troubleshooting skills across networking, compute, storage, and containers. Preferred Certifications: Azure DevOps Engineer Expert / AWS Certified DevOps Engineer Professional / Google Professional DevOps Engineer HashiCorp Certified: Terraform Associate (Preferred for Terraform users) Show more Show less

Requirements Proficient in SQL and Linux with hands-on experience. Strong understanding of the Hadoop ecosystem and job scheduling tools like Airflow and Oozie. Skilled in writing and executing SQL queries for comprehensive data validation. Familiarity with test automation frameworks (e. g., Robot Framework), with automation skills as an asset. Basic programming knowledge in Python is a plus. Experience with S3 buckets and cloud storage workflows is advantageous. Strong analytical and problem-solving skills with a high attention to detail. Excellent verbal and written communication abilities. Ability to collaborate effectively in a fast-paced Agile/Scrum environment. Adaptable and eager to learn new tools, technologies, and processes. 2-5 years of experience in Big Data testing, focusing on both automated and manual testing for data validation and UI testing. Proven experience in testing Spark job performance, security, and integration across diverse systems. Hands-on experience with defect tracking tools such as JIRA or Bugzilla. This job was posted by Sushruti Nikumbh from Hoonartek. Show more Show less

Job Title: Principal Data Engineer (MTS4 / Principal Engineer) About the Role As a Principal Data Engineer, you will drive the strategy, architecture, and execution of large-scale data solutions across our function. This role involves tackling highly ambiguous, complex challenges where the business problem may not be fully defined at the outset. You will partner closely with cross-functional teams (Engineering, Product, Operations) to shape and deliver our data roadmap. Your work will have a profound impact on our functions' data capabilities, influencing multiple teams’ technical and product direction. You should bring deep expertise in designing and developing robust data pipelines and platforms, leveraging technologies such as Spark, Airflow, Kafka, and other emerging tools. You will set standards and best practices that raise the bar for engineering excellence across the organization. Key Responsibilities Architect & Define Scope Own end-to-end design of critical data pipelines and platforms in an environment characterized by high ambiguity. Translate loosely defined business objectives into a clear technical plan, breaking down complex problems into achievable milestones. Technology Leadership & Influence Provide thought leadership in data engineering, driving the adoption of Spark, Airflow, Kafka, and other relevant technologies (e.g., Hadoop, Flink, Kubernetes, Snowflake, etc.). Lead design reviews and champion best practices for coding, system architecture, data quality, and reliability. Influence senior stakeholders (Engineers, EMs, Product Managers) on technology decisions and roadmap priorities. Execution & Delivery Spearhead strategic, multi-team projects that advance the organization’s data infrastructure and capabilities. Deconstruct complex architectures into simpler components that can be executed by various teams in parallel. Drive operational excellence, owning escalations and ensuring high availability, scalability, and cost-effectiveness of our data solutions. Mentor and develop engineering talent, fostering a culture of collaboration and continuous learning. Impact & Technical Complexity Shape how the organization operates by introducing innovative data solutions and strategic technical direction. Solve endemic, highly complex data engineering problems with robust, scalable, and cost-optimized solutions. Continuously balance short-term business needs with long-term architectural vision. Process Improvement & Best Practices Set and enforce engineering standards that elevate quality and productivity across multiple teams. Lead by example in code reviews, automation, CI/CD practices, and documentation. Champion a culture of continuous improvement, driving adoption of new tools and methodologies to keep our data ecosystem cutting-edge. Qualifications Education & Experience : Bachelor’s or Master’s degree in Computer Science, Engineering, or related field (or equivalent experience). 5+ years of software/data engineering experience, with significant exposure to large-scale distributed systems. Technical Expertise : Demonstrated proficiency with Spark, Airflow, Kafka, and at least one major programming language (e.g., Python, Scala, Java). Experience with data ecosystem technologies such as Hadoop, Flink, Snowflake, Kubernetes, etc. Proven track record of architecting and delivering highly scalable data infrastructure solutions. Leadership & Communication : Ability to navigate and bring clarity in ambiguous situations. Strong cross-functional collaboration skills, influencing both technical and non-technical stakeholders. Experience coaching and mentoring senior engineers. Problem-Solving : History of tackling complex, ambiguous data challenges and delivering tangible results. Comfort making informed trade-offs between opportunity vs. architectural complexity. Show more Show less

Job Title: Principal Data Engineer (MTS4 / Principal Engineer) About the Role As a Principal Data Engineer, you will drive the strategy, architecture, and execution of large-scale data solutions across our function. This role involves tackling highly ambiguous, complex challenges where the business problem may not be fully defined at the outset. You will partner closely with cross-functional teams (Engineering, Product, Operations) to shape and deliver our data roadmap. Your work will have a profound impact on our functions' data capabilities, influencing multiple teams’ technical and product direction. You should bring deep expertise in designing and developing robust data pipelines and platforms, leveraging technologies such as Spark, Airflow, Kafka, and other emerging tools. You will set standards and best practices that raise the bar for engineering excellence across the organization. Key Responsibilities Architect & Define Scope Own end-to-end design of critical data pipelines and platforms in an environment characterized by high ambiguity. Translate loosely defined business objectives into a clear technical plan, breaking down complex problems into achievable milestones. Technology Leadership & Influence Provide thought leadership in data engineering, driving the adoption of Spark, Airflow, Kafka, and other relevant technologies (e.g., Hadoop, Flink, Kubernetes, Snowflake, etc.). Lead design reviews and champion best practices for coding, system architecture, data quality, and reliability. Influence senior stakeholders (Engineers, EMs, Product Managers) on technology decisions and roadmap priorities. Execution & Delivery Spearhead strategic, multi-team projects that advance the organization’s data infrastructure and capabilities. Deconstruct complex architectures into simpler components that can be executed by various teams in parallel. Drive operational excellence, owning escalations and ensuring high availability, scalability, and cost-effectiveness of our data solutions. Mentor and develop engineering talent, fostering a culture of collaboration and continuous learning. Impact & Technical Complexity Shape how the organization operates by introducing innovative data solutions and strategic technical direction. Solve endemic, highly complex data engineering problems with robust, scalable, and cost-optimized solutions. Continuously balance short-term business needs with long-term architectural vision. Process Improvement & Best Practices Set and enforce engineering standards that elevate quality and productivity across multiple teams. Lead by example in code reviews, automation, CI/CD practices, and documentation. Champion a culture of continuous improvement, driving adoption of new tools and methodologies to keep our data ecosystem cutting-edge. Qualifications Education & Experience : Bachelor’s or Master’s degree in Computer Science, Engineering, or related field (or equivalent experience). 5+ years of software/data engineering experience, with significant exposure to large-scale distributed systems. Technical Expertise : Demonstrated proficiency with Spark, Airflow, Kafka, and at least one major programming language (e.g., Python, Scala, Java). Experience with data ecosystem technologies such as Hadoop, Flink, Snowflake, Kubernetes, etc. Proven track record of architecting and delivering highly scalable data infrastructure solutions. Leadership & Communication : Ability to navigate and bring clarity in ambiguous situations. Strong cross-functional collaboration skills, influencing both technical and non-technical stakeholders. Experience coaching and mentoring senior engineers. Problem-Solving : History of tackling complex, ambiguous data challenges and delivering tangible results. Comfort making informed trade-offs between opportunity vs. architectural complexity. Show more Show less

Job Title: AI/ML Engineer Location: Pune, India About the Role: We’re looking for highly analytical, technically strong Artificial Intelligence/Machine Learning Engineers to help build scalable, data-driven systems in the digital marketing space. You'll work alongside a top-tier team on impactful solutions affecting billions of users globally. Experience Required: 3 – 7 years Key Responsibilities: Collaborate across Data Science, Ops, and Engineering to tackle large-scale ML challenges. Build and manage robust ML pipelines (ETL, training, deployment) in real-time environments. Optimize models and infrastructure for performance and scalability. Research and implement best practices in ML systems and lifecycle management. Deploy deep learning models using high-performance computing environments. Integrate ML frameworks into cloud/distributed systems. Required Skills: 2+ years of Python development in a programming-intensive role. 1+ year of hands-on ML experience (e.g., Classification, Clustering, Optimization, Deep Learning). 2+ years working with distributed frameworks (Spark, Hadoop, Kubernetes). 2+ years with ML tools such as TensorFlow, PyTorch, Keras, MLlib. 2+ years experience with cloud platforms (AWS, Azure, GCP). Excellent communication skills. Preferred: Prior experience in AdTech or digital advertising platforms (DSP, Ad Exchange, SSP). Education: M.Tech or Ph.D. in Computer Science, Software Engineering, Mathematics, or a related discipline. Why Apply? Join a fast-moving team working on the forefront of AI in advertising. Build technologies that impact billions of users worldwide. Shape the future of programmatic and performance advertising. Show more Show less

RandomTrees is a leading Data & AI company offering a diverse range of products and services within the data and AI space. We are seeking a skilled Big Data Engineer. As a strategic partner of IBM, we support multiple industries, including Pharma, Banking, Semiconductor, Oil & Gas, and more. Additionally, we are actively engaged in research and innovation in Generative AI (GenAI) and Conversational AI. Headquartered in the United States, we also have offices in Hyderabad and Chennai, India. Job Title: Big Data Engineer Experience: 5-9 Years Location: Hyderabad-Hybrid Employment Type: Full-Time Job Summary: We are seeking a skilled Big Data Engineer with 5-9 years of experience in building and managing scalable data pipelines and analytics solutions. The ideal candidate will have strong expertise in Big Data, Hadoop, Apache Spark, SQL, Hadoop, and Data Lake/Data Warehouse architectures. Experience working with any cloud platform (AWS, Azure, or GCP) is preferred. Required Skills: 59 years of hands-on experience as a Big Data Engineer. Strong proficiency in Apache Spark (PySpark or Scala). Solid understanding and experience with SQL and database optimization. Experience with data lake or data warehouse environments and architecture patterns. Good understanding of data modeling, performance tuning, and partitioning strategies. Experience in working with large-scale distributed systems and batch/stream data processing. Preferred Qualifications: Experience with cloud platforms preferably GCP or AWS, Azure. Education: Bachelors degree in Computer Science, Engineering, or a related field.

What you will do In this vital role We are seeking a highly skilled and hands-on Senior Software Engineer Search to drive the development of intelligent, scalable search systems across our pharmaceutical organization. You'll work at the intersection of software engineering, AI, and life sciences to enable seamless access to structured and unstructured contentspanning research papers, clinical trial data, regulatory documents, and internal scientific knowledge. This is a high-impact role where your code directly accelerates innovation and decision-making in drug development and healthcare delivery Design, implement, and optimize search services using technologies such as Elasticsearch, OpenSearch, Solr, or vector search frameworks. Collaborate with data scientists and analysts to deliver data models and insights. Develop custom ranking algorithms, relevancy tuning, and semantic search capabilities tailored to scientific and medical content Support the development of intelligent search features like query understanding, question answering, summarization, and entity recognition Build and maintain robust, cloud-native APIs and backend services to support high-availability search infrastructure (e.g., AWS, GCP, Azure Implement CI/CD pipelines, observability, and monitoring for production-grade search systems Work closely with Product Owners, Tech Architect. Enable indexing of both structured (e.g., clinical trial metadata) and unstructured (e.g., PDFs, research papers) content Design & develop modern data management tools to curate our most important data sets, models and processes, while identifying areas for process automation and further efficiencies Expertise in programming languages such as Python, Java, React, typescript, or similar. Strong experience with data storage and processing technologies (e.g., Hadoop, Spark, Kafka, Airflow, SQL/NoSQL databases). Demonstrate strong initiative and ability to work with minimal supervision or direction Strong experience with cloud infrastructure (AWS, Azure, or GCP) and infrastructure as code like Terraform In-depth knowledge of relational and columnar SQL databases, including database design Expertise in data warehousing concepts (e.g. star schema, entitlement implementations, SQL v/s NoSQL modeling, milestoning, indexing, partitioning) Experience in REST and/or GraphQL Experience in creating Spark jobs for data transformation and aggregation Experience with distributed, multi-tiered systems, algorithms, and relational databases. Possesses strong rapid prototyping skills and can quickly translate concepts into working code Develop and execute unit tests, integration tests, and other testing strategies to ensure the quality of the software Analyze and understand the functional and technical requirements of applications Identify and resolve software bugs and performance issues Work closely with multi-functional teams, including product management, design, and QA, to deliver high-quality software on time Maintain detailed documentation of software designs, code, and development processes Basic Qualifications: Degree in computer science & engineering preferred with 6-8 years of software development experience Proficient in Databricks, Data engineering, Python, Search algorithms using NLP/AI models, GCP Cloud services, GraphQL Hands-on experience with search technologies (Elasticsearch, Solr, OpenSearch, or Lucene). Hands on experience with Full Stack software development. Proficient in programming languages, Java, Python, Fast Python, Databricks/RDS, Data engineering, S3Buckets, ETL, Hadoop, Spark, airflow, AWS Lambda Experience with data streaming frameworks (Apache Kafka, Flink). Experience with cloud platforms (AWS, Azure, Google Cloud) and related services (e.g., S3, Redshift, Big Query, Databricks) Hands on experience with various cloud services, understand pros and cons of various cloud services in well architected cloud design principles Working knowledge of open-source tools such as AWS lambda. Strong problem solving, analytical skills; Ability to learn quickly; Excellent communication and interpersonal skills Preferred Qualifications: Experience in Python, Java, React, Fast Python, Typescript, JavaScript, CSS HTML is desirable Experienced with API integration, serverless, microservices architecture. Experience in Data bricks, PySpark, Spark, SQL, ETL, Kafka Solid understanding of data governance, data security, and data quality best practices Experience with Unit Testing, Building and Debugging the Code Experienced with AWSAzure Platform, Building and deploying the code Experience in vector database for large language models, Databricks or RDS Experience with DevOps CICD build and deployment pipeline Experience in Agile software development methodologies Experience in End-to-End testing Experience in additional Modern Database terminologies. Good to Have Skills Willingness to work on AI Applications Experience in MLOps, React, JavaScript, Java, GCP Search Engines Experience with popular large language models Experience with LangChain or LlamaIndex framework for language models Experience with prompt engineering, model fine tuning Knowledge of NLP techniques for text analysis and sentiment analysis Soft Skills: Excellent analytical and troubleshooting skills. Strong verbal and written communication skills. Ability to work effectively with global teams. High degree of initiative and self-motivation. Team-oriented, with a focus on achieving team goals. Strong presentation and public speaking skills. We will ensure that individuals with disabilities are provided with reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.

Greetings from TCS!!! TCS has been a great pioneer in feeding the fire of young Techies like you. We are a global leader in the technology arena and there’s nothing that can stop us from growing together. Experience - 5+ years Location - Pune • Expertise with Big Data, Hadoop ecosystem, Spark(Scala / Java). • Experience in working with large cloud data lakes. • Experience with large-scale data processing, complex event processing, stream processing. • Experience in working with CI/CD pipelines, source code repositories, and operating environments. • Experience in working with both structured and unstructured data, with a high degree of SQL knowledge. • Experience designing and implementing scalable ETL/ELT processes and modeling data for low latency reporting • Experience in performance tuning, troubleshooting and diagnostics, process monitoring, and profiling. • Understanding containerization, virtualization, and cloud computing. • Object-oriented programming and component-based development with Java. • Experience working in the Scrum Agile software development framework. • Ability to work in a fast-paced environment with evolving requirements and capability goals. Show more Show less

Position: Solution Architect Location: Chennai/ Bangalore/ Kuala Lumpur Experience: 8+ years Employment Type: Full-time Job Overview Join Moving Walls, a trailblazer in the Out-of-Home (OOH) advertising and AdTech ecosystem, as a Solution Architect. This pivotal role places you at the heart of our innovative journey, designing and implementing scalable, efficient, and transformative solutions for our award-winning platforms like LMX and MAX . With a focus on automating and enhancing media transactions, you’ll enable a seamless connection between media buyers and sellers in a rapidly evolving digital-first landscape. As a Solution Architect, you will bridge the gap between business objectives and technical execution, working in an Agile environment with POD-based execution models to ensure ownership and accountability. You will drive initiatives that revolutionize the way data and technology shape OOH advertising. Why Join Us? ● Innovative Vision: Be part of a team committed to "Creating the Future of Outernet Media", where every solution impacts global markets across Asia, ANZ, Africa, and more. ● Cutting-edge Projects: Work on features like programmatic deal automation, data-driven audience insights, and dynamic campaign management for platforms connecting billions of ad impressions. ● Collaborative Culture: Collaborate with multidisciplinary teams, including Sales, Product Management, and Engineering, to craft solutions that are customized and impactful. What You’ll Do: ● Architect scalable and innovative solutions for AdTech products, ensuring alignment with organizational goals and market needs. ● Collaborate with cross-functional teams to gather, analyze, and translate business requirements into technical designs. ● Lead the development of programmatic solutions, dynamic audience segmentation tools, and integrations for global markets. ● Enhance existing products by integrating advanced features like dynamic rate cards, bid management, and inventory mapping. ● Advocate for best practices in system design, ensuring the highest standards of security, reliability, and performance. What You Bring: ● A strong technical background with hands-on experience in cloud-based architectures, API integrations, and data analytics. ● Proven expertise in working within an Agile environment and leading POD-based teams to deliver high-impact results. ● Passion for AdTech innovation and the ability to navigate complex, fast-paced environments. ● Excellent problem-solving skills, creativity, and a customer-centric mindset. Key Responsibilities 1. Solution Design: ○ Develop end-to-end solution architectures for web, mobile, and cloud-based platforms using the specified tech stack. ○ Translate business requirements into scalable and reliable technical solutions. 2. Agile POD-Based Execution: ○ Collaborate with cross-functional POD teams (Product, Engineering, QA, and Operations) to deliver iterative and focused solutions. ○ Ensure clear ownership of deliverables within the POD, fostering accountability and streamlined execution. ○ Contribute to defining and refining the POD stages to ensure alignment with organizational goals. 3. Collaboration and Stakeholder Management: ○ Work closely with product, engineering, and business teams to define technical requirements. ○ Lead technical discussions with internal and external stakeholders. 4. Technical Expertise: ○ Provide architectural guidance and best practices for system integrations, APIs, and microservices. ○ Ensure solutions meet non-functional requirements like scalability, reliability, and security. 5. Documentation: ○ Prepare and maintain architectural documentation, including solution blueprints and workflows. ○ Create technical roadmaps and detailed design documentation. 6. Mentorship: ○ Guide and mentor engineering teams during development and deployment phases. ○ Review code and provide technical insights to improve quality and performance. 7. Innovation and Optimization: ○ Identify areas for technical improvement and drive innovation in solutions. ○ Evaluate emerging technologies to recommend the best tools and frameworks. Required Skills and Qualifications ● Bachelor’s/Master’s degree in Computer Science, Information Technology, or a related field. ● Proven experience as a Solution Architect or a similar role. ● Expertise in programming languages and frameworks: Java, Angular, Python, C++ ● Proficiency in AI/ML frameworks and libraries such as TensorFlow, PyTorch, Scikit-learn, or Keras. ● Experience in deploying AI models in production, including optimizing for performance and scalability. ● Understanding of deep learning, NLP, computer vision, or generative AI techniques. ● Hands-on experience with model fine-tuning, transfer learning, and hyperparameter optimization. ● Strong knowledge of enterprise architecture frameworks (TOGAF, Zachman, etc.). ● Expertise in distributed systems, microservices, and cloud-native architectures. ● Experience in API design, data pipelines, and integration of AI services within existing systems. ● Strong knowledge of databases: MongoDB, SQL, NoSQL. ● Proficiency in working with large-scale datasets, data wrangling, and ETL pipelines. ● Hands-on experience with CI/CD pipelines for AI development. ● Version control systems like Git and experience with ML lifecycle tools such as MLflow or DVC. ● Proven track record of leading AI-driven projects from ideation to deployment. ● Hands-on experience with cloud platforms (AWS, Azure, GCP) for deploying AI solutions. ● Familiarity with Agile methodologies, especially POD-based execution models. ● Strong problem-solving skills and ability to design scalable solutions. ● Excellent communication skills to articulate technical solutions to stakeholders. Preferred Qualifications ● Experience in e-commerce, Adtech or OOH (Out-of-Home) advertising technology. ● Knowledge of tools like Jira, Confluence, and Agile frameworks like Scrum or Kanban. ● Certification in cloud technologies (e.g., AWS Solutions Architect). Tech Stack ● Programming Languages: Java, Python or C++ ● Frontend Framework: Angular ● Database Technologies: MongoDB, SQL, NoSQL ● Cloud Platform: AWS ● Familiarity with data processing tools like Pandas, NumPy, and big data frameworks (e.g., Hadoop, Spark). ● Experience with cloud platforms for AI (AWS SageMaker, Azure ML, Google Vertex AI). ● Understanding of APIs, microservices, and containerization tools like Docker and Kubernetes. Share your profile to kushpu@movingwalls.com Show more Show less