Only a solid grounding in computer engineering, Unix, data structures and algorithms would enable you to meet this challenge. 7+ years of experience architecting, developing, releasing, and maintaining large-scale big data platforms on AWS or GCP Understanding of how Big Data tech and NoSQL stores like MongoDB, HBase/HDFS, ElasticSearch synergize to power applications in analytics, AI and knowledge graphs Understandingof how data processing models, data location patterns, disk IO, network IO, shuffling affect large scale text processing - feature extraction, searching etc Expertise with a variety of data processing systems, including streaming, event, and batch (Spark, Hadoop/MapReduce) 5+ years proficiency in configuring and deploying applications on Linux-based systems 5+ years of experience Spark - especially Pyspark for transforming large non-structured text data, creating highly optimized pipelines Experience with RDBMS, ETL techniques and frameworks (Sqoop, Flume) and big data querying tools (Pig, Hive) Stickler of world class best practices, uncompromising on the quality of engineering, understand standards and reference architectures and deep in Unix philosophy with appreciation of big data design patterns, orthogonal code design and functional computation models Skills:- Apache Hadoop, PySpark, Python, Design patterns, Data Structures and Algorithms
Role Overview: As a Market Analytics QC at DemandMatrix, your primary responsibility will be to generate and validate technographics data, product keywords, and track upcoming technologies. You are expected to possess a deep understanding of various technologies, be able to differentiate similar technology products based on nuanced features, and conduct competitive analysis effectively. Key Responsibilities: - Understand business objectives, specifically for marketing and sales teams of technology product vendors. Design research methods to discover data sources and validate technographics data generated by our pipelines. - Collect data on technology products, vendors, consumers, competitors, and marketplace. Consolidate information into actionable items, reports, and presentations to make recommendations. - Compile and analyze data using modern and traditional methods. Utilize online market research and catalog findings into databases. - Evaluate program methodology and key data to ensure accuracy and correctness of the release angle. - Stay fully informed on market trends, other parties" research, and implement best practices. Qualifications: - Proven market research analysis experience for technology products and their vendors. - Ability to interpret large amounts of data and multitask effectively. - Strong communication and presentation skills. - Excellent knowledge of SQL, MS Excel, and basic Python scripting. - Proficiency in search engines, web analytics, and business research tools. - Familiarity with CRM programs and data collection methods. - Working knowledge of data warehousing, modeling, and mining. - Strong analytical and critical thinking skills. - BS degree in Engineering, MBA with a technical background related to Data.,