Data Engineering Engineer

Position Description:

Employees in this job function are responsible for designing, building, and maintaining data solutions including data infrastructure, pipelines, etc. for collecting, storing, processing and analyzing large volumes of data efficiently and accurately

Key Responsibilities:

1) Collaborate with business and technology stakeholders to understand current and future data requirements

2) Design, build and maintain reliable, efficient and scalable data infrastructure for data collection, storage, transformation, and analysis

3) Plan, design, build and maintain scalable data solutions including data pipelines, data models, and applications for efficient and reliable data workflow

4) Design, implement and maintain existing and future data platforms like data warehouses, data lakes, data lakehouse etc. for structured and unstructured data

5) Design and develop analytical tools, algorithms, and programs to support data engineering activities like writing scripts and automating tasks

6) Ensure optimum performance and identify improvement opportunities

Skills Required:

Python, Dataproc, dataflow, GCP Cloud Run, Agile Software Development, DataForm, TERRAFORM, Big Query, Data Fusion, GCP, Cloud SQL, KAFKA

Experience Required:

Bachelor's degree in Computer Science, Engineering, or a related technical field 5+ years of SQL development experience 5+ years of analytics/data product development experience required 3+ years of Google cloud experience with solutions designed and implemented at production scale Experience working in GCP native (or equivalent) services like Big Query, Google Cloud Storage, Dataflow, Dataproc, etc 2+ Experience working with Airflow for scheduling and orchestration of data pipelines 1+ Experience working with Terraform to provision Infrastructure as Code 2 + years professional development experience in Python

Experience Preferred:

In-depth understanding of Google's product technology (or other cloud platform) and underlying architectures Experience with development eco-system such as Tekton/Cloud Build, Git Experience in working with DBT/Dataform

Education Required:

Bachelor's Degree

Additional Information :

You will work on ingesting, transforming, and analyzing large datasets to support the Enterprise Securitization Solution Experience with large scale solution and operationalization of data lakes, data warehouses, and analytics platforms on Google Cloud Platform or other cloud environments is a must Work in collaborative environment that leverages paired programming Work on a small agile team to deliver curated data products Work effectively with product owners, data champions and other technical experts Demonstrate technical knowledge and communication skills with the ability to advocate for well-designed solutions Develop exceptional analytical data products using both streaming and batch ingestion patterns on Google CloudPlatform with solid data warehouse principles Be the Subject Matter Expert in Data Engineering with a focus on GCP native services and other well integrated third-party technologies