GCP Data engineer -II

Software Engineer-II engineers secure, scalable and reliable technology solutions, with minimal mentoring, to advance CMEG in the global marketplace and serve risk management needs of customers around the world.

Key Responsibilities:

• Provide operational support for our analytics system built on the KDB+ database management system and tooling. This includes monitoring performance, troubleshooting issues, and maintaining data pipelines to ensure system stability.
• Actively participate in the design, development, and implementation of new analytics solutions on GCP. This will involve refactoring existing workflows and building new capabilities using native GCP services.
• Utilize GCP's real-time analytics stack to build robust and scalable data pipelines. This includes leveraging Pub/Sub for messaging and Dataflow or similar services for data processing.
• Develop and maintain dashboards and reports using Looker/Tableau to provide business users with actionable insights. This involves defining LookML models and creating visualizations.
• Work closely with data scientists, product managers, and other engineering teams to understand requirements and deliver high-quality analytics solutions

Principal Accountabilities:

Required Skills & Qualifications:

• GCP Analytics: Proven experience with GCP's analytics services, including BigQuery, Pub/Sub, and Looker.
• KDB+ & Q language: A minimum level of proficiency with KDB+ and the Q language to perform daily maintenance and troubleshooting. This is not a development-heavy role, but you must be able to "keep the lights on" for our existing systems.
• SQL Expertise: Strong proficiency in SQL is essential for querying, data manipulation, and performance tuning in BigQuery.
• Data Modeling: Experience with data modeling techniques for both traditional and cloud-based data warehouses.
• Scripting/Programming: Familiarity with a scripting language like Python or Bash for automation and data pipeline management.
• Problem-Solving: Excellent analytical and problem-solving skills with a keen attention to detail.

Preferred Qualifications:

• Experience with other GCP services like Cloud Functions, Cloud Storage, or VPC.

• Knowledge of distributed systems and real-time data processing architectures.
• Familiarity with financial data and trading systems.
• Experience with version control systems, such as Git.