291 Web Scraping Jobs

Role & responsibilities Job Description - IT Junior Executive. The Junior IT Executive will provide technical support to warehouse sta and assist in managing the IT that supports daily warehouse operations. This individual will handle troubleshooting support the implementation of new technologies, ensuring eicient and warehouse processes. Responsibilities Provide IT support: Serve as the first point of contact for warehouse sta, addressing and resolving issues with warehouse technology, barcode scanners and handheld devices. Manage systems: Assist with the upkeep of the Warehouse Management System (WMS) and other key software platforms. This includes routine updates and user account manageme...

About The Role We are seeking a skilled and detail-oriented Data Engineer to join our team You will be responsible for designing, developing, and maintaining scalable ETL pipelines, optimizing data infrastructure, and ensuring data integrity through governance best practices This role involves working closely with quantitative teams and developers to build robust data solutions while leveraging cutting-edge big data technologies and cloud platforms Key Responsibilities Design, develop, and maintain scalable ETL pipelines for multi-scale datasets Architect and optimize high-performance data pipelines for seamless data flow and transformation Implement data quality checks and governance proces...

As a Google Cloud & Web Scraping Engineering EXPERT, you will be part of a team that is not just another real estate company, but an automation company focused on building a data infrastructure to revolutionize the real estate market. Your role will involve deploying scraping clusters across 100+ Google Cloud instances, optimizing cost and throughput, and constructing anti-fragile automation pipelines. You will have the opportunity to architect and deploy high-scale web scrapers using Python, Playwright, and Google Cloud Platform (GCP), manage distributed queues, proxies, and CAPTCHA bypass, monitor costs, speed, and data integrity across large pipelines, and collaborate directly with founde...

About The Role We are seeking a skilled and detail-oriented Data Engineer to join our team You will be responsible for designing, developing, and maintaining scalable ETL pipelines, optimizing data infrastructure, and ensuring data integrity through governance best practices This role involves working closely with quantitative teams and developers to build robust data solutions while leveraging cutting-edge big data technologies and cloud platforms Key Responsibilities Design, develop, and maintain scalable ETL pipelines for multi-scale datasets Architect and optimize high-performance data pipelines for seamless data flow and transformation Implement data quality checks and governance proces...

The Team: The Automotive Insights - Supply Chain and Technology and IMR department at S&P Global is dedicated to delivering critical intelligence and comprehensive analysis of the automotive industry's supply chain and technology. Our team provides actionable insights and data-driven solutions that empower clients to navigate the complexities of the automotive ecosystem, from manufacturing and logistics to technological innovations and market dynamics. We collaborate closely with industry stakeholders to ensure our research supports strategic decision-making and drives growth within the automotive sector. Join us to be at the forefront of transforming the automotive landscape with cutting-ed...

Role Overview: You will be responsible for developing the Policies & Next Gen Rule Engine for Digital Lending. Additionally, you will need to develop various RestFul APIs & integrate them with various data sources. Another key aspect of your role will involve developing various reports & automating tasks required for the business. Your proficiency in programming using Python & DB will be crucial for this role. Moreover, having experience in Numpy & Pandas will be beneficial. Key Responsibilities: - Develop Policies & Next Gen Rule Engine for Digital Lending - Develop various RestFul APIs & integrate with different data sources - Develop various reports & automate tasks necessary for the busi...

As a Senior Automation Tester - Network Security at ACL Digital, you will be responsible for ensuring End-to-End Solution testing in the Security SDWAN area. Your role will involve utilizing your expertise in network security concepts, protocols, and technologies to contribute to the QA process. Below are the key responsibilities and qualifications required for this position: **Key Responsibilities:** - Hands-on experience in End-to-End Solution testing in Security SDWAN area - Strong understanding of network security concepts, protocols, and technologies - QA experience with VPN technologies such as IKev2, IKev1, IPSec, SSL/TLS, etc. - QA experience with SD-WAN technologies and solutions - ...

Role Overview: As a Web Scraping Developer at our company, you will be responsible for designing, developing, and maintaining cutting-edge web scraping solutions to extract large datasets from various websites. Your role will be crucial in fueling our data-driven initiatives and ensuring the efficiency and accuracy of data extraction processes. Key Responsibilities: - Develop Cutting-Edge Solutions: Design, develop, and maintain robust web scraping solutions to extract large datasets. - Master Python Programming: Utilize advanced Python skills to implement and optimize sophisticated scraping scripts and tools. - Leverage Advanced Tools: Employ industry-leading tools such as BeautifulSoup, Sc...

As a Python Developer at Kicks Machine, your primary responsibility will be to design and maintain Python-based web scrapers to extract real-time product, price, and inventory data. You will be expected to clean, normalize, and transform scraped data using pandas for seamless ingestion. Additionally, you will integrate with Shopify's GraphQL and REST APIs to automate bulk product updates, inventory management, and price adjustments. Your role will also involve building and managing automated workflows and Google Apps Script for daily KPI dashboards and order imports. Collaboration with marketing, operations, and finance teams is essential to ensure data accuracy and support data-driven decis...

We are seeking a Python Developer with strong Linux skills and hands-on experience in PDF parsing and data extraction. The candidate should be self-driven and capable of delivering efficient POCs and production-ready code with minimal supervision. Responsibilities: Develop and maintain Python utilities and automation scripts on Linux. Build and optimize PDF parsing pipelines using tools like Tabula, PyMuPDF, or PDFPlumber. Extract and structure data from unstructured PDFs using Pandas and Regex. Manage system-level tasks, shell scripts, and services with systemd. Debug, optimize, and enhance existing codebases. Requirements: 3+ years of Python development experience. 3+ years working on Linu...

About the Role We are looking for a skilled Python Developer to join our development team. You will be responsible for designing, developing, and maintaining scalable backend applications and APIs using Python. The ideal candidate is passionate about writing clean, efficient code and enjoys solving complex technical challenges. Key Responsibilities Design, develop, and maintain backend applications and RESTful APIs using Python . Collaborate with front-end developers, product managers, and other stakeholders to define and implement technical solutions. Write clean, reusable, and efficient code following best practices. Integrate third-party APIs and data sources. Develop automated unit and i...

About the Role We are looking for a skilled Python Developer to join our development team. You will be responsible for designing, developing, and maintaining scalable backend applications and APIs using Python. The ideal candidate is passionate about writing clean, efficient code and enjoys solving complex technical challenges. Key Responsibilities Design, develop, and maintain backend applications and RESTful APIs using Python . Collaborate with front-end developers, product managers, and other stakeholders to define and implement technical solutions. Write clean, reusable, and efficient code following best practices. Integrate third-party APIs and data sources. Develop automated unit and i...

Role Overview: As a Data Analyst at our company, you will play a crucial role in collecting, processing, and analyzing data from various online sources to support IEPF share recovery. Your expertise in Python programming, web scraping, and data analysis will be essential in providing actionable insights for business decision-making. Key Responsibilities: - **Data Scraping:** Utilize Python and web scraping techniques to extract data from financial, regulatory, and shareholding-related websites for IEPF share recovery. - **Data Cleaning & Preprocessing:** Clean, process, and structure raw data to ensure data quality and integrity for analysis purposes. - **Data Analysis & Visualization:** Ana...

As a highly skilled GenAI & Agentic AI Consultant, you will contribute to the rapid development of the LLM-powered SaaS platform. This remote, contract-based opportunity focuses on delivering real-world AI solutions with LLM, Agentic AI, LangGraph, and GenAI stacks. - Lead the development and delivery of MVP and critical AI modules such as the blog engine, AI dashboard, and automation system. - Architect and integrate reusable modules including RSS scrapers, rewrite engines, and token-optimized AI APIs. - Collaborate with cross-functional teams like Product, Design, DevOps, and QA. - Utilize LLMs, LangChain, LangGraph, and AI orchestration frameworks like Agentic AI and MCP. - Maintain Agile...

Role Overview: As a Consultant in the National Assurance team at EY, you will be part of the Audit Standards and Methodologies unit, contributing to inspiring confidence and trust in the business world. Your role will involve ensuring compliance with audit standards, providing clear perspectives to audit committees, and offering critical information to stakeholders. You will have the opportunity to work on various service offerings such as External Audit, Financial Accounting Advisory Services, IFRS & US GAAP conversion, and more. Key Responsibilities: - Develop automated analytics models for statutory audit enablements - Conduct sector-specific analytics using advanced/custom analytics tool...

About this role: Wells Fargo is seeking a Securities Research Associate In this role, you will: Provide support in the editing and publishing functions within the Global Research and Economics Group in Wells Fargo Securities Generate research packet products, post research documents to market data vendors, update and maintain mailing lists, and provide client access to web sites Assign reports to Editors and Supervisory Analysts to ensure the most efficient use of resources Maintain databases Receive direction from leaders Exercise independent judgment while developing the knowledge to understand function, policies, procedures, and compliance requirements Collaborate and consult with peers, ...

Role Overview: As an Associate Consultant-National-Assurance-ASU in Audit Standards and Methodologies at EY, you will be part of a global leader in assurance, tax, transaction, and advisory services. EY believes in providing you with the training, opportunities, and creative freedom to help you build a better working world. You will have the chance to work across industries for a well-rounded experience and contribute to inspiring confidence and trust in the business world. Key Responsibilities: - Demonstrate technical excellence in audit analytics - Conduct sector-specific analytics using advanced/custom analytics - Utilize visualization tools for data representation - Develop automated ana...

What You'll Do Design, build, and maintain the Scrapingdog API layer and its dashboard infrastructure (Node.js, Express, React) Optimise and scale our MongoDB-backed data models for speed and reliability Implement robust job queues, caching layers, and credit metering logic Build and maintain CI/CD pipelines for automated deployments Deploy and manage services on DigitalOcean (droplets) Improve monitoring, observability, and fault tolerance across our stack Develop and maintain web scraping modules using Puppeteer, Playwright, and headless browsers Collaborate with the infrastructure, browser, and product teams to bring new scraping features to life Help evolve our architecture to support 10...

Job Summary: Hands-on Python developer to support the backend development of the MDM analytics platform. This role involves building and maintaining data processing pipelines, integrating APIs, and working with the senior architect to implement technical solutions. Familiarity with GraphQL, REST APIs, and data modeling is expected. Key Responsibilities: •Develop backend services and data connectors in Python. •Support GraphQL API development and testing. •Assist in integrating with MDM and external data sources. •Write unit tests and maintain code quality standards. •Collaborate with DevOps and architects for deployment readiness. Preferred Skills: Experience with asynchronous programming (A...

Company Overview BOLD (A parent company of Career Builder and Monster US) is an established and fast-growing product company that transforms work lives. Since 2005,weve helped more than 10,000,000 folks from all over America(and beyond!) reach higher and do better. A career at BOLD promises great challenges, opportunities, culture and the environment. With our headquarters in Puerto Rico and offices in San Francisco and India, were a global organization on a path to change the career industry. Position Overview BOLD (A parent company of Career Builder and Monster US) is seeking a highly skilled professional to spearhead the development of cutting-edge browser automation technology for our Ex...

Overview Stats Perform is the market leader in sports tech. We provide the most trusted sports data to some of the world's biggest organizations, across sports, media, and broadcasting. Through the latest AI technologies and machine learning, we combine decades' worth of data with the latest in-game happenings. We then offer coaches, teams, professional bodies, and media channels around the world, access to the very best data, content, and insights. In turn, improving how sports fans interact with their favorite sports teams and competitions. How do they use it? Media outlets add a little magic to their coverage with our stats and graphics packages. Sportsbooks can offer better predictions a...

Requirements for Python Developer (Healthcare RCM Automation) Technical Skills and Experience : Minimum 4-6 years of professional experience with below skills. Advanced Python Programming : Extensive experience in Python, with a deep understanding of Python principles, design patterns, and best practices. Proficiency in developing scalable and efficient Python code, with a focus on automation, data processing, and backend services. Demonstrated ability with automation libraries like PyAutoGUI for GUI automation tasks, enabling the automation of mouse and keyboard actions. Experience with Selenium for web automation , capable of automating web browsers to mimic user actions, scrape web data, ...

Role & responsibilities As a Data Engineer focused on web crawling and platform data acquisition, you will design, develop, and maintain large-scale web scraping pipelines to extract valuable platform data. You will be responsible for implementing scalable and resilient data extraction solutions, ensuring seamless data retrieval while working with proxy management, anti-bot bypass techniques, and data parsing. Optimizing scraping workflows for performance, reliability, and efficiency will be a key part of your role. Additionally, you will ensure that all extracted data maintains high quality and integrity. Preferred candidate profile We are seeking candidates with: Strong experience in Pytho...

As a Backend Python Django Developer at our company, you will play a crucial role in designing, developing, and maintaining server-side applications using the Django Rest Framework. Your expertise will be instrumental in creating efficient, scalable, and robust backend systems that power our web applications. Responsibilities: - Develop, deploy, and maintain server-side logic using Django and Django Rest Framework. - Collaborate with Frontend developers to integrate APIs. - Implement data storage solutions using databases such as MsSQL, MySQL, or PostgreSQL. - Troubleshoot and resolve issues, proposing solutions and implementing fixes. - Design and implement REST APIs using Django REST Frame...

Building and maintaining advanced web scraping and automation scripts across platforms (LinkedIn, Facebook, Twitter, Discord). Handle CAPTCHA, proxy, IP rotation, and anti-blocking. Ensure deployable, scalable, and monitored solutions using Docker.