Job
Description
We are seeking a skilled Data Engineer to join our dynamic team at Game Theory. This role is pivotal in building and maintaining our data infrastructure, ensuring the efficient collection, storage, processing, and analysis of data related to sports games and events. The successful candidate will work closely with cross-functional teams to drive data-driven decision-making and enhance our sports technology solutions, including preparing high-quality datasets to power our AI models. Key Responsibilities: Data Pipeline Development : Design, develop, and maintain robust ETL pipelines to process sports game data, including scores, computer vision analysis, skill scores, win/loss ratios, streaks, and leaderboards. NoSQL Database Management : Create and optimize MongoDB collections, indexes, and aggregation pipelines to ensure efficient data storage and retrieval. Data Integration : Integrate various data sources, ensuring data consistency and reliability across the platform. Database Management : Create and manage database schemas for both SQL and NoSQL databases, optimize query performance, and ensure data integrity. Data Cleaning and Preparation : Perform data cleaning, transformation, and aggregation to prepare datasets for analysis, reporting, and AI model training. AI Model Data Preparation : Develop processes to transform raw data into structured formats suitable for machine learning models, including feature engineering and dataset balancing. Metrics Generation : Develop and maintain metrics and dashboards that provide insights into player performance, game statistics, and other key indicators. Data Quality Assurance : Implement data quality checks and monitoring processes to ensure data accuracy and completeness. Collaboration : Work closely with data scientists, analysts, and software engineers to understand data needs and deliver solutions that meet business requirements. Documentation and Best Practices : Document data processes, pipelines, and best practices to ensure maintainability and reproducibility. Required Qualifications: 2-4 years as a Data Engineer, preferably within a tech startup or sports technology company. Strong proficiency in SQL and experience with relational databases (eg, PostgreSQL, MySQL). Experience with NoSQL databases, particularly MongoDB, including schema design, aggregation framework, and performance optimization. Experience with ETL tools and frameworks (eg, Apache Airflow, Apache NiFi). Proficiency in programming languages such as Python or Scala. Familiarity with data modelling, data warehousing, and big data technologies (eg, Hadoop, Spark). Knowledge of data preparation techniques for machine learning applications. Strong analytical and problem-solving skills with attention to detail. Excellent communication and collaboration skills. Bachelors degree in Computer Science, Engineering, or a related field. Preferred Qualifications: Experience with cloud-based data platforms (eg, AWS, GCP, Azure). Knowledge of sports data and metrics, including ELO ratings, win/loss ratios, and leaderboards. Experience with data visualization tools. Understanding of AI/ML workflows and the specific data requirements for training models. Experience with data streaming technologies (eg, Kafka, Kinesis). Knowledge of data governance and compliance requirements. Masters degree in a related field. What We Offer: The opportunity to be at the forefront of sports technology innovation. A dynamic and challenging environment that promotes personal and professional growth. Autonomy to steer projects with a significant impact on the sports industry. Competitive compensation package, including equity options
