448 Scrapy Jobs - Page 12

Location: [Remote / India] Job Type: [Full-time] Experience: 3+ years in web crawling/scraping, backend systems, and data extraction About Client Our Client is a modern, meaning‑based web-search API designed specifically for AI applications—such as retrieval‑augmented generation (RAG). Unlike traditional keyword-based engines, they use embedding-based semantic search, allowing developers to fetch content that’s contextually relevant and up-to-date About the Role: We are looking for a skilled Web Crawler Engineer to design, develop, and maintain scalable web crawling and scraping systems. The ideal candidate should be well-versed in handling large-scale data extraction, parsing unstructured w...

About Us: We are an innovative and data-driven organization committed to harnessing the power of data to drive strategic decisions, efficiency, and automation. Our mission is to transform complex data into actionable insights, fueling growth and innovation. Position Overview: We are seeking a skilled Web Scraper & Data Automation Specialist to join our dynamic data operations team. The successful candidate will play a pivotal role in extracting, structuring, and automating data retrieval from diverse sources, ensuring its accuracy, relevance, and usability for internal analytics and strategic projects. Responsibilities: Design, develop, and maintain web scraping scripts and automation tools ...

Job Title :: Python Developer Location :: Gurugram, India (Hybrid) Minimum 4 years of relevant experience Job Description: Strong understanding of Python and web scraping techniques, with experience in frameworks such as Scrapy or Selenium. Knowledge of various Python libraries, APIs, and toolkits, including hands-on experience with Pandas, databases, and SQL server. Proficient in data extraction methods (e.g., PDF data extraction, Excel automation) and code versioning tools (e.g., Git). Skilled in writing scalable code, testing, and debugging applications to ensure quality and functionality. Experience in developing back-end components and integrating data storage solutions while optimizing...

About The Role We are seeking a highly skilled Web Scraping & Python API Developer to build and maintain scalable data extraction systems from various websites and APIs. The ideal candidate has hands-on experience with web scraping frameworks, RESTful API development, and data integration techniques. Responsibilities Design and develop robust, scalable web scraping scripts using Python (e.g., Scrapy, BeautifulSoup, Selenium). Build and maintain RESTful APIs to serve scraped data to internal systems or clients. Handle anti-bot mechanisms like CAPTCHAs, JavaScript rendering, and IP rotation. Optimize scraping processes for speed, reliability, and data integrity. Parse and normalize structured ...

As a Python Web Scraping & AI Engineer at ZANG - an AI-powered E-commerce Search Engine based in Mumbai, you will play a crucial role in revolutionizing the e-commerce landscape by contributing to the development of cutting-edge web scraping infrastructure and AI algorithms. ZANG is a dynamic startup that is focused on innovation, and we are seeking a skilled individual like you to enhance our search capabilities. Your primary responsibilities will include designing, building, and maintaining scalable web scraping solutions using Python to extract structured data from various e-commerce websites. You will be tasked with handling challenges such as CAPTCHA, rotating proxies, and anti-scraping...

Arrk Group India is a technology company with over 23 years of industry experience, specializing in developing scalable platforms to meet the needs of customers and enabling organizations to implement disruptive technologies for business transformation. The focus is on helping both public and private sector customers globally transition to digital environments. As a Senior AI Engineer at Arrk Group India, you will be responsible for AI Engineering System Architecture and Development. This includes understanding generative AI, large language models, and other foundation models for various business applications. You should possess demonstrable knowledge and practical experience in using models...

Job Title: Data Analyst – D2C eCommerce Location: Gurgaon, Haryana Experience: 3+ Years (Mandatory experience in D2C eCommerce industry) About the Role: We are looking for an experienced Data Analyst with a strong background in the Direct-to-Consumer (D2C) eCommerce industry . The ideal candidate will have proven expertise in web scraping , data processing , and visualization tools to derive actionable insights. This role demands a proactive thinker who understands eCommerce metrics, consumer behavior, and performance tracking. Key Responsibilities: Collect, clean, and process large datasets from various eCommerce platforms, CRMs, and online sources. Perform web scraping to gather competitor...

Job Title: Data Analyst – D2C eCommerce Location: Gurgaon, Haryana Experience: 3+ Years (Mandatory experience in D2C eCommerce industry) About the Role: We are looking for an experienced Data Analyst with a strong background in the Direct-to-Consumer (D2C) eCommerce industry . The ideal candidate will have proven expertise in web scraping , data processing , and visualization tools to derive actionable insights. This role demands a proactive thinker who understands eCommerce metrics, consumer behavior, and performance tracking. Key Responsibilities: Collect, clean, and process large datasets from various eCommerce platforms, CRMs, and online sources. Perform web scraping to gather competitor...

Spherex is seeking an Artificial Intelligence (AI) Machine Learning (ML) Engineer to contribute to the development, enhancement, and expansion of our product platform catering to the Media and Entertainment sector. As the AI/ML Engineer, your duties will involve the creation of machine learning models and system retraining. The position is based in Navi Mumbai, India. The ideal candidate should hold a degree in computer science or software development. Proficiency in .Net, Azure, Project management, Team and Client management is essential. Additionally, familiarity with Python, Tensorflow, Pytorch, MySQL, Artificial Intelligence, and Machine Learning is desired. Key requirements for this rol...

As a backend developer, you will be responsible for designing and developing scalable backend systems using Python, specifically Django, Flask, or FastAPI. You will also be tasked with building data pipelines and implementing data extraction techniques using tools like Selenium, Scrapy, or APIs. In addition, you will create and maintain RESTful APIs for data integration and frontend communication. Your role will also involve building responsive and interactive frontend interfaces using React.js, Vue.js, or similar frameworks. Furthermore, you will work with large datasets and integrate data workflows into full stack applications. Collaboration with designers and product teams to convert UI/U...

About Us: Headquartered in Sunnyvale, with offices in Dallas & Hyderabad, Fission Labs is a leading software development company, specializing in crafting flexible, agile, and scalable solutions that propel businesses forward. With a comprehensive range of services, including product development, cloud engineering, big data analytics, QA, DevOps consulting, and AI/ML solutions, we empower clients to achieve sustainable digital transformation that aligns seamlessly with their business goals. Fission Labs Website: https://www.fissionlabs.com/ Work Location: Hyderabad Notice Period: Immediate to 30 Days Role Overview: Omada is on a mission to build next-gen intelligent systems that integrate re...

About Cisive Cisive is a trusted partner for comprehensive, high-risk compliance-driven background screening and workforce monitoring solutions, specializing in highly regulated industries—such as healthcare, financial services, and transportation. We catch what others miss, and we are dedicated to helping our clients effortlessly secure the right talent. As a global leader, Cisive empowers organizations to hire with confidence. Through our PreCheck division, Cisive provides specialized background screening and credentialing solutions tailored for healthcare organizations, ensuring patient and workforce safety. Driver iQ, our transportation-focused division, delivers FMCSA-compliant screenin...

Job Title: Python Web Scraper & UI/UX Designer Location: Chennai Employment Type: Full-time About the Role: We are building lean, fast, and user-focused applications. This role blends two key capabilities: Python-based web scraping and UI/UX design . The ideal candidate can both automate the extraction of web data and design the user interface through which that data is presented. This is an application-facing role, meaning the scraping logic you build and the designs you create will be integrated directly into the products we ship. You will work closely with both developers and product owners to ensure seamless user experiences backed by live or regularly updated data. Responsibilities: Web...

Urgent Opening: Web Scraping – Data Crawling, AI/ML Location: Permanent Work From Home Job Type: Full-Time | Permanent Experience: 1.5+ Years (Preferred) About the Role We are looking for a skilled and experienced Python Developer with strong expertise in data crawling, web scraping, AI/ML, and CAPTCHA solving techniques. The ideal candidate is passionate about automation, data pipelines, and problem-solving with a deep understanding of the web ecosystem. This is a permanent remote opportunity, ideal for professionals looking to work in a flexible and innovative environment while delivering high-quality solutions in data acquisition and intelligent automation. Key Responsibilities Design and...

About the Role: Grade Level (for internal use): 08 Job Title: Associate Data Engineer Location: Bangalore (Hybrid) The Team: The Automotive Insights - Supply Chain and Technology and IMR department at S&P Global is dedicated to delivering critical intelligence and comprehensive analysis of the automotive industry's supply chain and technology. Our team provides actionable insights and data-driven solutions that empower clients to navigate the complexities of the automotive ecosystem, from manufacturing and logistics to technological innovations and market dynamics. We collaborate closely with industry stakeholders to ensure our research supports strategic decision-making and drives growth wi...

As a member of the Data Engineering team, your primary responsibility will be to handle various aspects of data extraction. This includes understanding the data requirements of the business group, reverse-engineering the website and its technology, developing web robots to automate data extraction, and building monitoring systems to ensure data integrity. You will also play a key role in managing changes to website dynamics and layout for clean downloads, creating scraping and parsing systems to structure raw data, and providing operations support for high availability and zero data losses. Moreover, you will be involved in storing extracted data in recommended databases, constructing scalab...

You will be responsible for developing cutting-edge solutions by designing, developing, and maintaining robust web scraping solutions to extract large datasets from various websites, supporting our data-driven initiatives. Your role will involve mastering Python programming to implement and optimize sophisticated scraping scripts and tools. You will leverage industry-leading tools such as BeautifulSoup, Scrapy, Selenium, and other scraping frameworks to efficiently collect and process data. Additionally, you will innovate with AI technologies like ChatGPT to automate and enhance data extraction processes, pushing the boundaries of what is possible. It will be crucial to optimize data managem...

About Rivan Solutions Rivan Solutions is a web scraping and web designing company. We are based in Hyderabad but we serve the entire world. About the work from home job/internship Selected intern's day-to-day responsibilities include: 1. Scraping websites using Python and BeautifulSoup/Scrapy or Selenium 2. Delivering the web scraping script within 6 to 7 hours. 3. Working with Python and Beautiful Soup to scrape and automate the websites 4. Storing data into SQL and NoSQL 5. Being regular to meetings and delivering on time Preferred Passouts on or before 2025 are preferred. Skill(s) required JSON, Python, SQL Who can apply Only those candidates can apply who: 1. are available for the work f...

Responsibilities Develop Cutting-Edge Solutions : Design, develop and maintain robust web scraping solutions that extract large datasets from various websites, fueling our data-driven initiatives. Master Python Programming : Utilize advanced Python skills to implement and optimize sophisticated scraping scripts and tools. Leverage Advanced Tools : Employ industry-leading tools such as BeautifulSoup, Scrapy, Selenium, and other scraping frameworks to collect and process data efficiently. Innovate with AI : Use ChatGPT prompt skills to automate and enhance data extraction processes, pushing the boundaries of whats possible. Optimize Data Management : Clean, organize, and store extracted data i...

Setting up LAMP environment and hosting and configuring php projects. Troubleshooting network, processes, disk related issues. Installation and configuration of Samba and NFS. Setting up routers/modems and linux firewalls (iptables). Configuring MySql replication and clusters. Installation and configuration of PXE server for installing Linux over the Network. Technical writing skills producing clear and unambiguous technical documentation and user stories. Writing shell script to automate the tasks. Developing spiders using Scrapy framework (python) for crawling and scrapping the web. Troubleshooting issues of php scripts and supporting web developers to develop websites. Experience of perfo...

Data Science Intern Location: Remote Duration: 2-6 months Type: Unpaid Internship About Collegepur Collegepur is an innovative platform dedicated to providing students with comprehensive information about colleges, career opportunities, and educational resources. We are building a dynamic team of talented individuals passionate about data-driven decision-making. Job Summary We are seeking a highly motivated Data Science Intern to join our team. This role involves working on data collection, web scraping, analysis, visualization, and machine learning to derive meaningful insights that enhance our platform’s functionality and user experience. Responsibilities: Web Scraping: Collect and extract...

Job Title: AI Engineer – Web Crawling & Field Data Extraction Location: [Remote] Department: Engineering / Data Science Experience Level: Mid to Senior Employment Type: Contract to Hire About the Role: We are looking for a skilled AI Engineer with strong experience in web crawling, data parsing, and AI/ML-driven information extraction to join our team. You will be responsible for developing systems that automatically crawl websites, extract structured and unstructured data, and intelligently map the extracted content to predefined fields for business use. This role combines practical web scraping, NLP techniques, and AI model integration to automate workflows that involve large-scale content...

Software Engineer - Content Parsing The Opportunity We're looking for a talented and detail-oriented Software Engineer - Content Parsing to join our dynamic team. In this role, you'll be crucial in extracting, categorizing, and structuring vast amounts of content from various web sources. You'll leverage your expertise in Python and related parsing technologies to build robust, scalable, and highly accurate parsing solutions. This position requires a strong focus on data quality, comprehensive testing, and the ability to implement effective alerting and notification systems. Responsibilities : Design, develop, and maintain robust and scalable HTML parsing solutions to extract diverse web con...

Responsibilities Design and build the core backend architecture using Python, Node.js, or similar stacks. Create and manage scalable databases (PostgreSQL, MongoDB, etc). Develop scraping engines to gather artist, curator, and event data. Integrate secure contract logic and payment flows using APIs or Web3 tools. Work closely with the product and frontend teams to co-create seamless user experiences. Take full ownership of APIs, hosting, and technical decisions. Requirements 1 to 3 years of backend or data engineering experience, or strong independent projects. Good Grip over MERN Stack and Next.js Framework. Confidence with JavaScript and frameworks like Flask, Django, Node, and Express. St...

Hiring Senior Data Engineer for an AI-native startup. Work on scalable data pipelines, LLM workflows, web scraping (Scrapy, lxml), Pandas, APIs, and Django. Strong in Python, data quality, mentoring, and large-scale systems. Health insurance