Senior Machine Learning Engineer

1. Project description

Migration of User Knowledge Objects from Splunk to Clickhouse / Grafana / Bosun:

PRE Observability team is undertaking a large-scale observability transformation initiative aimed at migrating from Splunk to a more cost-effective and scalable open-source observability stack based on Clickhouse and Grafana. As part of this effort, we are seeking POD-based teams to support the migration of user knowledge objects such as dashboards, reports, alerts, macros and lookups.

2. Client description

3.

• Lead the design, development, and deployment of AI-based automation tools for data artifact migration.
• Develop machine learning models to intelligently map, transform, and validate data across different big data platforms.
• Build robust data pipelines to handle high-volume, high-velocity data migration.
• Collaborate with data engineers and architects to integrate AI-driven solutions into existing data workflows.
• Implement NLP and pattern recognition algorithms to automate schema conversion and data validation.
• Design custom algorithms for automated data quality checks and anomaly detection.
• Mentor junior engineers and contribute to technical leadership within the AI engineering team.
• Stay updated with the latest advancements in AI, big data technologies, and automation frameworks.
• Create comprehensive technical documentation and best practice guidelines for AI-based data migration.

4. Details on tech stack

• Splunk
• ClickHouse
• Grafana
• Data Migration Automation