103 Web Crawling Jobs

Role Overview: As a Web Scraper, your main responsibility is to utilize your expertise in fetching data from various online sources. You will be tasked with developing efficient web scrapers and parsers for different websites, extracting both structured and unstructured data, and storing them in SQL/NoSQL databases. Additionally, you will collaborate closely with Project, Business, and Research teams to provide them with the scraped data for analysis. It is crucial to maintain the scraping projects that are delivered to production, develop frameworks to automate data flow from multiple sources, and work independently with minimal supervision. You are expected to have a deep understanding of ...

Technical Skills (Mandatory) Strong hands-on experience with TypeScript Backend development using Node.js Web scraping using Puppeteer / Playwright / Selenium Strong understanding of HTML, CSS, DOM, and JavaScript execution Experience with HTTP requests, REST APIs, Cookies, Headers, Sessions Data handling with JSON, CSV, and databases (MongoDB / MySQL / PostgreSQL) Experience with Git / Version Control Kriti Madan | Recruitment Lead People Staffing Solutions kriti.madan@peoplestaffing.in | 9354235524

Purpose/Objective The Lead – Store Inventory Management in the Techno Commercial Department will be responsible for managing the airport’s inventory and store operations with a focus on cost optimization, vendor coordination, and commercial compliance. This role bridges technical requirements with commercial execution, ensuring efficient material handling and procurement support for airport operations. Key Responsibilities of Role - Inventory & Store Management:- Oversee receipt, storage, and issuance of materials and equipment. - Maintain accurate inventory records using ERP systems and ensure timely reconciliation. - Define SOPs and guidelines for store management, inventory control and sc...

Purpose/Objective To Manage and handle warehouse operations activities in Manufacturing plant with the following requirement : 1) First hand experience in Store and logistics. 2)Demonstrate methodical and precise approach towards commercial and contractual information of Store and logistics 3)Exclusive knowledge on Solar business / Infrastructure O&M / project and well integrated with all functions. Key Responsibilities of Role Scrap Management of Non BOM ,BOM and Modules Scrap for Solar Manufacturing Business Overall Scrap Inventory Management and Reconciliation. Timely disposal of scrap without any accumulation. Adherence of Scrap SOP Adherence to Safety Compliances Desired Area of Experti...

We are seeking a Web Crawling Engineer who will be responsible for building and maintaining web crawlers, extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms. About Forage AI : Forage AI is a pioneering AI-powered data extraction and automation company that transforms complex, unstructured web and document data into clean, structured intelligence. Our platform combines web crawling, NLP, LLMs, and agentic AI to deliver highly accurate firmographic and enterprise insights across numerous domains. Trusted by ...

Summary To enhance user profiling and risk assessment, we are building web crawlers to collect relevant user data from third-party sources, forums, and the dark web. We are seeking a Senior Web Crawler & Data Extraction Engineer to design and implement these data collection solutions. Job Responsibilities Design, develop, and maintain web crawlers and scrapers to extract data from open web sources, forums, marketplaces, and the dark web. Implement data extraction pipelines that aggregate, clean, and structure data for fraud detection and risk profiling. Use Tor, VPNs, and other anonymization techniques to safely crawl the dark web while avoiding detection. Develop real-time monitoring soluti...

As a Machine Learning Engineer with 9-12 years of experience, you will play a key role in designing and deploying agentic AI architectures involving LLMs, Agentic AI frameworks, and MCP-based architectures. Your responsibilities will include: - Agentic AI & LLM Development: - Design, implement, and optimize agentic workflows using LangChain, LangGraph, n8n, and related orchestration tools. - Develop RAG solutions by leveraging vector databases, relational databases, and MongoDB. - Implement web crawling and external MCP services like Tavily to enhance agent capabilities. - Knowledge Base Engineering: - Build knowledge repositories from text, audio, and video sources using embeddings, transcr...

As a Backend Python Developer at our company, you will be responsible for designing, developing, and maintaining scalable backend applications using Python and modern frameworks. Your role will involve building and optimizing RESTful APIs and microservices with FastAPI, as well as developing and maintaining robust web crawling and scraping solutions using Python libraries and Selenium for automation. Additionally, you will implement and manage containerized applications using Docker and orchestrate them with Kubernetes. Integration and management of relational and non-relational databases (MS SQL Server and MongoDB) will also be a key part of your responsibilities. Collaboration with cross-f...

Posting description We are seeking a skilled Python professional to build robust data pipelines, integrate diverse APIs (including Large Language Models), extract critical insights from various data sources, and automate data reporting to drive key business decisions Shift Pattern A : 12:30 PM - 10:00 PM Shift Pattern B : 2:30 PM - 12:00 AM Responsibilities We are seeking a skilled Python professional to build robust data pipelines, integrate diverse APIs (including Large Language Models), extract critical insights from various data sources, and automate data reporting to drive key business decisions. API & LLM Integration: Develop and maintain Python solutions for secure and efficient integ...

Role & responsibilities As a Data Engineer focused on web crawling and platform data acquisition, you will design, develop, and maintain large-scale web scraping pipelines to extract valuable platform data. You will be responsible for implementing scalable and resilient data extraction solutions, ensuring seamless data retrieval while working with proxy management, anti-bot bypass techniques, and data parsing. Optimizing scraping workflows for performance, reliability, and efficiency will be a key part of your role. Additionally, you will ensure that all extracted data maintains high quality and integrity. Preferred candidate profile We are seeking candidates with: Strong experience in Pytho...

Role Overview: As an Associate- Database Management at Innovaccer, you will work closely with the Marketing team to manage and optimize marketing databases, run data analysis, and develop targeted marketing campaigns to enhance customer engagement, retention, and growth. Your role will involve collaborating with cross-functional teams to ensure data-driven, personalized, and efficient marketing efforts. Key Responsibilities: - Maintain, update, and optimize customer databases to ensure accuracy and cleanliness for marketing campaigns - Manage customer segmentation to customize marketing efforts for specific audience segments - Design and execute targeted email, SMS, and digital marketing cam...

Product Name : Applyblink, Product Description: The product is in HR Tech domain 1. About the Role : We are looking for a talented and motivated Full Stack Developer with strong expertise in React.js, Node.js, PostgreSQL, and Chrome Extension development to join our engineering team. The ideal candidate should be capable of working across the full stack, building browser-based automation tools, developing efficient backend APIs, and creating high-performing user interfaces. This role involves end-to-end feature ownershipcontributing to architecture design, backend development, frontend implementation, automation workflows, extension development, testing, deployment, and production support. Y...

**Role Overview:** You will be responsible for developing the Policies and Next Gen Rule Engine for Digital Lending. Additionally, you will develop various RestFul APIs and integrate them with different data sources. Your role will also involve developing various reports and automating tasks required for the Business. **Key Responsibilities:** - Develop the Policies and Next Gen Rule Engine for Digital Lending - Develop various RestFul APIs and integrate them with various data sources - Develop various reports and automate tasks required for Business **Qualifications Required:** - Good Programming Skills using Python and DB - Good Experience in Numpy and Pandas - AWS or Other Cloud Platform ...

Role Overview: As an Associate - Database Management at Innovaccer, you will work closely with the Marketing team to manage and optimize marketing databases, run data analysis, and develop targeted marketing campaigns. Your role will involve maintaining accurate and clean data for marketing campaigns, managing customer segmentation, and collaborating with cross-functional teams to ensure data-driven and efficient marketing efforts. Key Responsibilities: - Maintain, update, and optimize customer databases for marketing campaigns - Manage customer segmentation to tailor marketing efforts to specific audience segments - Design and execute targeted email, SMS, and digital marketing campaigns bas...

We are seeking a skilled Web Scraping Developer with strong expertise in building scrapers, crawlers, and data extraction pipelines for e-commerce and quick-commerce platforms. The ideal candidate will have hands-on experience in large-scale product data extraction, including pricing, rankings, product descriptions, and catalog details. Key Responsibilities Design, develop, and maintain scalable scrapers/crawlers for e-commerce websites. Extract structured data such as product listings, pricing, descriptions, reviews, and rankings. Handle dynamic content, cookies, pagination, AJAX, and anti-bot challenges efficiently. Ensure data accuracy, integrity, and scalability of scraping pipelines. Op...

Role Overview: Our Go-To-Market (GTM) team at Innovaccer is passionate about making a positive impact in the healthcare ecosystem. As an Associate- Database Management, you will work closely with the Marketing team to manage and optimize marketing databases, run data analysis, and develop targeted marketing campaigns to enhance customer engagement, retention, and growth. Your role will involve collaborating with cross-functional teams to ensure that marketing efforts are data-driven, personalized, and efficient. Key Responsibilities: - Maintain, update, and optimize customer databases to ensure accurate and clean data for marketing campaigns - Manage customer segmentation to customize market...

Perform web scraping and automated data extraction from websites, directories, and online . Collect data related to vendors, venues, clients, markets, pricing, dcor elements, and competitors. Scrape images related to events, dcor styles, Required Candidate profile Experience scraping both text and images at scale. Ability to handle multiple projects and deliver clean, accurate data on time. Perks and benefits According to the Industry

Role & responsibilities Application Development: Design and build web-scale applications using Python (Django/Flask/FastAPI) and modern frontend frameworks (React / Angular / Vue). API Development: Develop and integrate RESTful and GraphQL APIs. Implement responsive, intuitive, and interactive user interfaces. Build scalable and secure backend systems with optimized database queries and caching strategies. Create and manage relational (PostgreSQL/MySQL) and NoSQL (MongoDB/Redis) databases. Deploy applications on AWS / Azure / GCP, including containerization (Docker, Kubernetes). Write clean, reusable, and maintainable code following industry best practices. Ensure system performance, securit...

About The Role Entity :- Accenture Strategy & Consulting Team :- Strategy & Consulting Global Network Practice :- Responsible AI COE Title :- Responsible AI Specialist/ Sr. Analyst Job location :- Bangalore/Gurgaon/Pune/ Hyderabad/Chennai/Mumbai The rapid development of AI is creating new opportunities to improve the lives of people around the world, from business to healthcare to education. As a result, it is also raising new questions about the best way to build fairness, interpretability, privacy and security into these systems. The Data and AI revolution is changing everything. Its everywhere transforming how we work and play. Join Accenture and help transform leading companies and commu...

About The Role Entity :- Accenture Strategy & Consulting Team :- Strategy & Consulting Global Network Practice :- Responsible AI COE Title :- Responsible AI Manager Job location :- Bangalore/Gurgaon/Pune/ Hyderabad/Chennai/Mumbai The rapid development of AI is creating new opportunities to improve the lives of people around the world, from business to healthcare to education. As a result, it is also raising new questions about the best way to build fairness, interpretability, privacy and security into these systems. The Data and AI revolution is changing everything. Its everywhere transforming how we work and play. Join Accenture and help transform leading companies and communities around th...

About The Role Entity :- Accenture Strategy & Consulting Team :- Strategy & Consulting Global Network Practice :- Responsible AI COE Title :- Responsible AI Specialist/ Sr. Analyst Job location :- Bangalore/Gurgaon/Pune/ Hyderabad/Chennai/Mumbai The rapid development of AI is creating new opportunities to improve the lives of people around the world, from business to healthcare to education. As a result, it is also raising new questions about the best way to build fairness, interpretability, privacy and security into these systems. The Data and AI revolution is changing everything. Its everywhere transforming how we work and play. Join Accenture and help transform leading companies and commu...

About The Role Entity :- Accenture Strategy & Consulting Team :- Strategy & Consulting Global Network Practice :- Responsible AI COE Title :- Responsible AI Specialist/ Sr. Analyst Job location :- Bangalore/Gurgaon/Pune/ Hyderabad/Chennai/Mumbai The rapid development of AI is creating new opportunities to improve the lives of people around the world, from business to healthcare to education. As a result, it is also raising new questions about the best way to build fairness, interpretability, privacy and security into these systems. The Data and AI revolution is changing everything. Its everywhere transforming how we work and play. Join Accenture and help transform leading companies and commu...

About ClearDemand: Clear Demand is the leader in AI-driven price and promotion optimization for retailers . Our platform transforms pricing from a challenge to a competitive advantage, helping retailers make smarter, data-backed decisions across the entire pricing lifecycle. By integrating competitive intelligence, pricing rules, and demand modelling , we enable retailers to maximize profit, drive growth, and enhance customer loyalty all while maintaining pricing compliance and brand integrity. With Clear Demand, retailers stay ahead of the market, automate complex pricing decisions, and unlock new opportunities for growth. Key Responsibilities: People Management Lead, coach, and develop a h...

Role & responsibilities Use AI and automation tools (e.g., ChatGPT, Perplexity, Phantombuster, Playwright, n8n, Zapier, or similar) to streamline data collection, cleaning, and validation. Perform data scraping, enrichment, and regular updates of law firm and attorney information. Identify process gaps and propose AI-based solutions to increase efficiency and accuracy. Work with the team to maintain structured data in spreadsheets, CRMs, and internal dashboards. Summarize and organize research findings clearly in written form. Collaborate with developers and analysts to test and improve automation workflows. Preferred candidate profile 13 years of experience in process operations research, o...

Entity :- Accenture Strategy & Consulting Team :- Strategy & Consulting Global Network Practice :- Responsible AI COE Title :- Responsible AI Specialist/ Sr. Analyst Job location :- Bangalore/Gurgaon/Pune/ Hyderabad/Chennai/Mumbai The rapid development of AI is creating new opportunities to improve the lives of people around the world, from business to healthcare to education. As a result, it is also raising new questions about the best way to build fairness, interpretability, privacy and security into these systems. The Data and AI revolution is changing everything. Its everywhere transforming how we work and play. Join Accenture and help transform leading companies and communities around t...