Job
Description
Data Acquisition & Web Application Developer Experience: 3 - 6 Years Exp Salary : USD 1,851-2,962 / month Preferred Notice Period : Within 30 Days Shift : 10:00AM to 7:00PM IST Opportunity Type: Remote Placement Type: Permanent (*Note: This is a requirement for one of Uplers' Clients) Must have skills required : APIS, data acquisition, Web scraping, Agile, Python Good to have skills : Analytics, Monitoring, stream processing, Web application deployment, Node Js GPRO Ltd (One of Uplers' Clients) is Looking for: Data Acquisition & Web Application Developer who is passionate about their work, eager to learn and grow, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then we want to hear from you. Role Overview Description Job Title: Data Acquisition & Web Application Developer About the Project: We are seeking a skilled full-stack developer to build a specialised web application designed to aggregate and present public information on individuals, such as company executives and leaders. This tool will serve as a comprehensive profile generator, pulling data from diverse online sources including news outlets, social media and other platforms. The primary goal is to provide users with a centralised, easily navigable view of a person's online presence, latest news and public information. Project Overview: The core of this project involves developing a robust data acquisition layer capable of scraping and integrating information from various online sources. This data will then be presented through a user-friendly web interface. The application should allow users to input a person's name and receive an aggregated view of relevant public data. Key Responsibilities: Develop and Implement Data Acquisition Layer: Design and build systems to scrape and collect data from specified sources, including news websites (e.g., Bloomberg.com, Reuters, BBC.com, Financial Times), social media (e.g., X, LinkedIn), and media platforms (e.g., YouTube, podcasts). Integrate with APIs: Utilize official APIs (e.g., Bloomberg data, Reuters, FinancialTimes, Google Finance) where available and prioritized. Evaluate and integrate with third-party scraping APIs (e.g., Apify, Oxylabs, SerpApi) as necessary, considering associated risks and subscription models. Handle Hybrid Approach: Implement a strategy that leverages licensed APIs for premium sources while potentially using third-party scrapers for others, being mindful of terms of service and legal/ethical considerations. Direct scraping of highly protected sites like Bloomberg, Reuters, and FT should be avoided or approached with extreme caution using third-party services. Design Data Storage and Indexing: Determine appropriate data storage solutions, considering the volume of data and its relevance over time. Implement indexing and caching mechanisms to ensure efficient search and retrieval of information, supporting near real-time data presentation. Develop Web Application Front-End: Build a basic, functional front-end interface similar to the provided examples ("Opening Screen," "Person profile"). This includes displaying the aggregated information clearly. Implement User Functionality: Enable users to: Input a person's name for searching. Sort displayed outputs by date. Click through links to access the original source of information. Navigate to a new search easily (e.g., via a tab). Consider Stream Processing: Evaluate and potentially implement stream processing techniques for handling near real-time data acquisition and updates. ¢ Ensure Scalability: Design the application to support a specified level of concurrent searches (estimated at 200 for the initial phase). ¢ Build Business Informational Layer: Develop a component that tracks the usage of different data services (APIs, scrapers) for monitoring costs and informing future scaling decisions. ¢ Technical Documentation: Provide clear documentation for the developed system, including data flows, API integrations, and deployment notes. Required Skills and Experience: ¢ Proven experience in web scraping and data acquisition from diverse online sources. ¢ Strong proficiency in developing with APIs, including handling different authentication methods and data formats. ¢ Experience with relevant programming languages and frameworks for web development and data processing (e.g., Python, Node.js, etc.). ¢ Knowledge of database design and data storage solutions. ¢ Familiarity with indexing and caching strategies for search applications. ¢ Understanding of potential challenges in web scraping (e.g., anti-scraping measures, terms of service). ¢ Experience in building basic web application front-ends. ¢ Ability to consider scalability and performance in system design. ¢ Strong problem-solving skills and ability to work independently or as part of a small team. ¢ Experience working with foreign (western based) startups and clients. Ability to work in agile environments and ability to pivot fast. Desirable Skills: ¢ Experience with stream processing technologies. ¢ Familiarity with deploying and managing web applications (though infrastructure design is flexible). ¢ Experience with monitoring and analytics for application usage. How to apply for this opportunity: Easy 3-Step Process: 1. Click On Apply! And Register or log in on our portal 2. Upload updated Resume & Complete the Screening Form 3. Increase your chances to get shortlisted & meet the client for the Interview! About Our Client: A web app aggregating real-time info on individuals for financial services professionals About Uplers: Our goal is to make hiring and getting hired reliable, simple, and fast. Our role will be to help all our talents find and apply for relevant product and engineering job opportunities and progress in their career. (Note: There are many more opportunities apart from this on the portal.) So, if you are ready for a new challenge, a great work environment, and an opportunity to take your career to the next level, don't hesitate to apply today. We are waiting for you!
