448 Scrapy Jobs - Page 13

TransForm Solutions is a trailblazer in the business process management and IT-enabled services industry, known for delivering top-notch solutions that drive business efficiency and growth. With a focus on innovation and excellence, the company empowers businesses to transform their operations and achieve their full potential. As the company continues to expand, they are looking for a dynamic Senior Web Data Scraping Engineer to join their team and help harness the power of data. Your mission in this role will involve developing cutting-edge solutions by designing, developing, and maintaining robust web scraping solutions that extract large datasets from various websites to fuel data-driven ...

The role involves both coding skills and broader engineering concepts like system design, API integration, and building complete workflows across multiple technology layers (frontend, backend, database, AI, and infrastructure). The responsibilities go beyond just writing code to include designing solutions and integrating various technologies into a cohesive system. Preferred Skills: Experience with AI frameworks including LLMs (ChatGPT, Claude, etc.), TensorFlow, PyTorch, or Hugging Face Familiarity with API development and integration, particularly for data extraction and processing Should have good experience with webscraping (Selenium, Scrapy, Playwright) Understanding of modern web deve...

eGrove Systems is looking for Lead Python Software Engineer to join its team of experts. Skill : Lead Python Software Engineer Exp: 5+Yrs NP: Immediate to 15Days Location: Chennai/Madurai Interested candidate can send your resume to annie@egrovesys.com Required Skills: - 5+ years of Strong experience in Python & 2 years in Django Web framework. Experience or Knowledge in implementing various Design Patterns. Good Understanding of MVC framework & Object-Oriented Programming. Experience in PGSQL / MySQL and MongoDB. Good knowledge in different frameworks, packages & libraries Django/Flask, Django ORM, Unit Test, NumPy, Pandas, Scrapy etc., Experience developing in a Linux environment, GIT & Ag...

We are hiring a tech-savvy and creative Social Media Handler with strong expertise in AI-powered content creation , web scraping , and automation of scraper workflows . You will be responsible for managing social media presence while automating content intelligence and trend tracking through custom scraping solutions. This is a hybrid role requiring both creative content skills and technical automation proficiency. Key Responsibilities: 1) Social Media Management - Plan and execute content calendars across platforms: Instagram, Facebook, YouTube, LinkedIn, and X. - Create high-performing, audience-specific content using AI tools (ChatGPT, Midjourney, Canva AI, etc.). - Engage with followers,...

About EQL Global EQL is redefining equity research with Al-powered tools that behave like a digital analyst. Our platform automates report writing, Q&A, document parsing and scenario modelling for analysts, brokers and investors. With tier-one financial institutions already on board, we're scaling fast and solving high-value problems in one of the world's most knowledge-intensive industries. Role Overview Join our distributed engineering team as a Backend & DevOps Engineer who bridges software development, cloud operations and information security. You will architect and maintain backend services, automate infrastructure, embed security controls, and drive our ISO 27001 compliance program. K...

eGrove Systems is looking for Senior Python Developer to join its team of experts. Skill : Senior Python Developer Exp : 5+Yrs NP : Immediate to 15Days Location : Chennai/Madurai Interested candidate can send your resume to annie@egrovesys.com Required Skills: - 5+ years of Strong experience in Python & 2 years in Django Web framework. Experience or Knowledge in implementing various Design Patterns. Good Understanding of MVC framework & Object-Oriented Programming. Experience in PGSQL / MySQL and MongoDB. Good knowledge in different frameworks, packages & libraries Django/Flask, Django ORM, Unit Test, NumPy, Pandas, Scrapy etc., Experience developing in a Linux environment, GIT & Agile metho...

As a Web Scrapers Developer (C2H) with 6-7 years of experience, you will excel in developing web scrapers to facilitate interactions with web pages. Your role will involve designing and implementing web scraping solutions to extract data from various web sources. You should possess strong programming skills in languages commonly used for web scraping, such as Python, JavaScript, or similar. Your expertise should include familiarity with web scraping libraries and frameworks like Beautiful Soup, Selenium, Scrapy, among others. It will be essential to automate data collection processes and ensure data quality through cleansing and validation. Additionally, you will prepare and transform data f...

We are seeking a highly skilled AI/ML Web Scraping Specialist to join our data engineering and analytics team. The ideal candidate will have hands-on experience in building robust web scraping pipelines focused on Instagram and other social media platforms (e.g., Facebook, TikTok, YouTube, X/Twitter) . The role involves designing scalable scraping architectures, solving anti-bot challenges, and applying machine learning to classify, enrich, or analyze social media content. Key Responsibilities Design and implement advanced web scraping tools, scripts, and pipelines using Python (BeautifulSoup, Scrapy, Selenium, Playwright, etc.). Build robust scrapers for social media platforms like Instagra...

About Cisive Cisive is a trusted partner for comprehensive, high-risk compliance-driven background screening and workforce monitoring solutions, specializing in highly regulated industries—such as healthcare, financial services, and transportation. We catch what others miss, and we are dedicated to helping our clients effortlessly secure the right talent. As a global leader, Cisive empowers organizations to hire with confidence. Through our PreCheck division, Cisive provides specialized background screening and credentialing solutions tailored for healthcare organizations, ensuring patient and workforce safety. Driver iQ, our transportation-focused division, delivers FMCSA-compliant screenin...

Python Developer – Web Scraping & Data Processing About the Role We are seeking a skilled and detail-oriented Python Developer with hands-on experience in web scraping, document parsing (PDF, HTML, XML), and structured data extraction. You will be part of a core team working on aggregating biomedical content from diverse sources, including grant repositories, scientific journals, conference abstracts, treatment guidelines, and clinical trial databases. Key Responsibilities • Develop scalable Python scripts to scrape and parse biomedical data from websites, pre-print servers, citation indexes, journals, and treatment guidelines. • Build robust modules for splitting multi-record documents (PDF...

Job Overview: We are looking for a highly skilled Python Developer with strong experience in building both web and mobile app crawlers. The ideal candidate should have in-depth knowledge of Scrapy, Selenium, REST APIs, proxy rotation, and techniques to bypass anti-scraping mechanisms. You'll be responsible for building scalable, stealthy crawlers that extract data from websites and mobile apps (via APIs or reverse engineering traffic). Responsibilities: Develop and maintain web crawlers using Scrapy, Selenium, and custom spiders. Build and manage mobile app crawlers by intercepting and decoding API requests, using tools like mitmproxy, Charles Proxy, or Burp Suite. Handle proxy rotation, use...

eGrove Systems is looking for Lead Django Backend Developer to join its team of experts. Skill : Lead Django Backend Developer Exp : 5+Yrs NP : Immediate to 15Days Location : Chennai/Madurai Interested candidate can send your resume to annie@egrovesys.com Required Skills: - 5+ years of Strong experience in Python & 2 years in Django Web framework. Experience or Knowledge in implementing various Design Patterns. Good Understanding of MVC framework & Object-Oriented Programming. Experience in PGSQL / MySQL and MongoDB. Good knowledge in different frameworks, packages & libraries Django/Flask, Django ORM, Unit Test, NumPy, Pandas, Scrapy etc., Experience developing in a Linux enviro...

You are someone who thrives in a full-stack Python + QA + Database + AI environment and is excited about technical growth. If you are a passionate Python engineer with hands-on experience in building web services, supporting QA/automation workflows, and working confidently across SQL/NoSQL databases, this opportunity is for you. Responsibilities: - Develop, test, and maintain Python-based backend services using Django, FastAPI, or Flask. - Execute efficient web scraping tasks using BeautifulSoup or Scrapy. - Automate browser workflows using Selenium. - Write scalable database queries and schema designs for SQL (PostgreSQL/MySQL) and MongoDB. - Design and implement QA automation scripts and f...

Role & responsibilities As a Data Engineer focused on web crawling and platform data acquisition, you will design, develop, and maintain large-scale web scraping pipelines to extract valuable platform data. You will be responsible for implementing scalable and resilient data extraction solutions, ensuring seamless data retrieval while working with proxy management, anti-bot bypass techniques, and data parsing. Optimizing scraping workflows for performance, reliability, and efficiency will be a key part of your role. Additionally, you will ensure that all extracted data maintains high quality and integrity. Preferred candidate profile We are seeking candidates with: Strong experience in Pytho...

As a Web Scraping & Data Automation Intern at D-Vivid Consultant, you will play a crucial role in developing, maintaining, and optimizing scripts to collect data from various platforms such as Instagram, Facebook, Reddit, and more. Your primary responsibilities will include utilizing automation tools and libraries like Make.com (Integromat), Python (BeautifulSoup, Scrapy, Selenium), and browser automation frameworks to ensure efficient data collection. You will be responsible for cleaning and organizing the extracted data to facilitate marketing intelligence and lead generation campaigns. Additionally, you will conduct research to enhance scraping practices and handle dynamic website content...

Director Data Science and Principal Data Scientist Experience range - 10+ years Type of job - Full time, In office Location - Bengaluru Roles And Responsibilities Convert broad vision and concepts into a structured data science roadmap, and guide a team to successfully execute on it. Handling end-to-end client AI & analytics programs in a fluid environment. Your role will be a combination of hands-on contribution, technical team management, and client interaction. Proven ability to discover solutions hidden in large datasets and to drive business results with their data-based insights Contribute to internal product development initiatives related to data science. Drive excellent project mana...

Job Description : Should have 4+ years hands-on experience in algorithms and implementation of analytics solutions in predictive analytics, text analytics and image analytics Should have handson experience in leading a team of data scientists, works closely with client’s technical team to plan, develop and execute on client requirements providing technical expertise and project leadership. Leads efforts to foster innovative ideas for developing high impact solutions. Evaluates and leads broad range of forward looking analytics initiatives, track emerging data science trends, and knowledge sharing Engaging key stakeholders to source, mine and validate data and findings and to confirm business...

Job Title: Python Developer – Data Extraction (Web Scraping) Experience: 4–5 Years Location: Concrete Software Solutions Pvt. Ltd., 9C, Techzone-4, Greater Noida West Job Type: Full-Time Company Overview Concrete Software Solutions Pvt. Ltd. is a leading IT services and product development company, providing innovative digital solutions across domains. We are currently expanding our data engineering team and looking for an experienced Python Developer with expertise in web scraping and data extraction , especially from grocery and e-commerce websites. Job Summary We are seeking a highly skilled and motivated Python Developer who specializes in data extraction and web scraping . The ideal can...

About Us At Zyte, we eat data for breakfast and you can eat your breakfast anywhere and work for Zyte. Founded in 2010, we are a globally distributed team of over 240 Zytans working from over 28 countries who are on a mission to enable our customers to extract the data they need to continue to innovate and grow their businesses. We believe that all businesses deserve a smooth pathway to data For more than a decade, Zyte has led the way in building powerful, easy-to-use tools to collect, format, and deliver web data, quickly, dependably, and at scale. And today, the data we extract helps thousands of organizations make smarter business decisions, secure competitive advantage, and drive sustai...

eGrove Systems is looking for Senior Python Backend Developer to join its team of experts. Skill : Senior Python Backend Developer Exp : 5+Yrs NP : Immediate to 15Days Location : Chennai/Madurai Interested candidate can send your resume to annie@egrovesys.com Required Skills: - 5+ years of Strong experience in Python & 2 years in Django Web framework. Experience or Knowledge in implementing various Design Patterns. Good Understanding of MVC framework & Object-Oriented Programming. Experience in PGSQL / MySQL and MongoDB. Good knowledge in different frameworks, packages & libraries Django/Flask, Django ORM, Unit Test, NumPy, Pandas, Scrapy etc., Experience developing in a Linux environment, G...

As a skilled Web Scraping Data Analyst, you will be responsible for collecting, cleaning, and analyzing data from various online sources. Your expertise in Python-based scraping frameworks, data transformation, and experience with proxy/VPN rotation and IP management will be crucial in building data pipelines that support our analytics and business intelligence initiatives. Your key responsibilities will include designing, developing, and maintaining robust web scraping scripts using tools like Python, BeautifulSoup, Scrapy, Selenium, etc. You will also implement IP rotation, proxy management, and anti-bot evasion techniques, deploy scraping tools on cloud-based or edge servers, and monitor ...

Job Title: Python Developer – Data Extraction (Web Scraping) Experience: 4–5 Years Location: Concrete Software Solutions Pvt. Ltd., 9C, Techzone-4, Greater Noida West Job Type: Full-Time Company Overview Concrete Software Solutions Pvt. Ltd. is a leading IT services and product development company, providing innovative digital solutions across domains. We are currently expanding our data engineering team and looking for an experienced Python Developer with expertise in web scraping and data extraction , especially from grocery and e-commerce websites. Job Summary We are seeking a highly skilled and motivated Python Developer who specializes in data extraction and web scraping . The ideal can...

We’re seeking a highly skilled Python Web Scraping Engineer with 3–4 years of hands-on experience in building robust data scrapers for a wide range of websites — from static pages to dynamic, JS-heavy platforms. The ideal candidate should have a deep understanding of Python-based scraping frameworks , web technologies, and anti-scraping defenses. Key Responsibilities: Design and build scalable scraping scripts using Python (Scrapy, BeautifulSoup, Selenium, Playwright) Extract structured and unstructured data from eCommerce, fashion, marketplace, and social media websites Handle dynamic content, lazy loading, and login-based content scraping Process and clean scraped data for AI models and da...

We’re Hiring: Data Scraping Engineer 📍 Location: [Remote/City, Country] | 🕒 Full-Time Techno Province is looking for a skilled and detail-oriented Data Scraping Engineer to join our growing tech team! You’ll be responsible for designing and developing robust web scraping solutions that help fuel powerful travel insights and automation across global platforms. 🔧 What You’ll Do: Build and maintain scalable web scraping tools and crawlers Extract data from travel websites, APIs, and unstructured sources Clean, structure, and store scraped data in usable formats (JSON, CSV, DB) Monitor scraping pipelines and troubleshoot issues (captcha, IP blocks, etc.) Work with developers and data teams to in...

As a skilled Web Scraping Data Analyst, your primary responsibility will involve collecting, cleaning, and analyzing data from various online sources. You will leverage your expertise in Python-based scraping frameworks to design, develop, and maintain robust web scraping scripts using tools such as Python, BeautifulSoup, Scrapy, Selenium, and more. Additionally, you will be tasked with implementing IP rotation, proxy management, and anti-bot evasion techniques to ensure efficient data collection. Your role will be instrumental in constructing data pipelines that drive our analytics and business intelligence initiatives. Collaboration will be a key aspect of your work as you engage with data...