Senior Data Engineer- GCP

The Senior Data Engineer engineers secure, scalable and reliable technology solutions, with minimal mentoring, to advance CMEG in the global marketplace and serve risk management needs of customers around the world.

Key Responsibilities:

• Provide operational support for our analytics system built on the KDB+ database management system and tooling. This includes monitoring performance, troubleshooting issues, and maintaining data pipelines to ensure system stability.
• Actively participate in the design, development, and implementation of new analytics solutions on GCP. This will involve refactoring existing workflows and building new capabilities using native GCP services.
• Utilize GCP's real-time analytics stack to build robust and scalable data pipelines. This includes leveraging Pub/Sub for messaging and Dataflow or similar services for data processing.
• Develop and maintain dashboards and reports using Looker/Tableau to provide business users with actionable insights. This involves defining LookML models and creating visualizations.
• Work closely with data scientists, product managers, and other engineering teams to understand requirements and deliver high-quality analytics solutions

Principal Accountabilities:

Required Skills & Qualifications:

• GCP Analytics: Proven experience with GCP's analytics services, including BigQuery, Pub/Sub, and Looker.
• KDB+ & Q language: A minimum level of proficiency with KDB+ and the Q language to perform daily maintenance and troubleshooting. This is not a development-heavy role, but you must be able to "keep the lights on" for our existing systems.
• SQL Expertise: Strong proficiency in SQL is essential for querying, data manipulation, and performance tuning in BigQuery.
• Data Modeling: Experience with data modeling techniques for both traditional and cloud-based data warehouses.
• Scripting/Programming: Familiarity with a scripting language like Python or Bash for automation and data pipeline management.
• Problem-Solving: Excellent analytical and problem-solving skills with a keen attention to detail.

Preferred Qualifications

• Experience with other GCP services like Cloud Functions, Cloud Storage, or VPC.
• Knowledge of distributed systems and real-time data processing architectures.
• Familiarity with financial data and trading systems.
• Experience with version control systems, such as Git.