Job
Description
As a Data Engineer at our company, your primary responsibilities will include: - Designing, developing, and maintaining scalable and efficient data pipelines using GCP services such as Dataflow, Cloud Composer (Airflow), and Pub/Sub. - Designing and implementing robust data models optimized for analytical and operational workloads within GCP data warehousing solutions like BigQuery. - Developing and implementing ETL (Extract, Transform, Load) processes to ingest, cleanse, transform, and load data from various sources into our data warehouse and other data stores on GCP. - Building and managing data warehousing solutions on GCP, ensuring data integrity, performance, and scalability. - Collaborating with data scientists and analysts to understand their data requirements and providing them with clean, transformed, and readily accessible data. - Monitoring and troubleshooting data pipelines and data warehousing systems to ensure data quality and availability. - Implementing data quality checks and validation processes to maintain the accuracy and reliability of data. - Optimizing data processing and storage for performance and cost-efficiency on the GCP platform. - Staying up-to-date with the latest GCP data services and best practices in data engineering and analytics. - Contributing to the documentation of data pipelines, data models, and ETL processes. - Working collaboratively with cross-functional teams to understand data needs and deliver effective data solutions. Qualifications: - Bachelor's degree in Computer Science, Engineering, or a related field. - Proven experience in data engineering principles and practices. - Solid understanding of data modeling concepts and techniques (e.g., relational, dimensional). - Hands-on experience in designing, developing, and implementing ETL processes. - Knowledge of data warehousing concepts, architectures, and best practices. - Familiarity with data analytics principles and the data lifecycle. - Strong problem-solving and analytical skills with the ability to troubleshoot data-related issues. - Excellent communication and collaboration skills. Preferred Qualifications (A Plus): - Hands-on experience working with Google Cloud Platform (GCP) data services such as BigQuery, Dataflow, Cloud Composer (Airflow), Pub/Sub, Cloud Storage, etc. - Experience with scripting languages such as Python or SQL for data manipulation and automation. - Familiarity with data visualization tools (e.g., Looker, Tableau). - Experience with data governance and data security principles. - Knowledge of DevOps practices and tools for data infrastructure automation. As a Data Engineer at our company, your primary responsibilities will include: - Designing, developing, and maintaining scalable and efficient data pipelines using GCP services such as Dataflow, Cloud Composer (Airflow), and Pub/Sub. - Designing and implementing robust data models optimized for analytical and operational workloads within GCP data warehousing solutions like BigQuery. - Developing and implementing ETL (Extract, Transform, Load) processes to ingest, cleanse, transform, and load data from various sources into our data warehouse and other data stores on GCP. - Building and managing data warehousing solutions on GCP, ensuring data integrity, performance, and scalability. - Collaborating with data scientists and analysts to understand their data requirements and providing them with clean, transformed, and readily accessible data. - Monitoring and troubleshooting data pipelines and data warehousing systems to ensure data quality and availability. - Implementing data quality checks and validation processes to maintain the accuracy and reliability of data. - Optimizing data processing and storage for performance and cost-efficiency on the GCP platform. - Staying up-to-date with the latest GCP data services and best practices in data engineering and analytics. - Contributing to the documentation of data pipelines, data models, and ETL processes. - Working collaboratively with cross-functional teams to understand data needs and deliver effective data solutions. Qualifications: - Bachelor's degree in Computer Science, Engineering, or a related field. - Proven experience in data engineering principles and practices. - Solid understanding of data modeling concepts and techniques (e.g., relational, dimensional). - Hands-on experience in designing, developing, and implementing ETL processes. - Knowledge of data warehousing concepts, architectures, and best practices. - Familiarity with data analytics principles and the data lifecycle. - Strong problem-solving and analytical skills with the ability to troubleshoot data-related issues. - Excellent communication and collaboration skills. Preferred Qualifications (A Plus): - Hands-on experience working with Google Cloud Platform (GCP) data services such as BigQuery, Dataflow, Cloud Composer (Airflow), Pub/Sub, Cloud Storage, etc. - Experience with scripting languages such as Pytho
