Job
Description
As a skilled Data Engineer with 6 to 8 years of experience, your role will involve designing, building, and maintaining data pipelines and data models that support analytical and business intelligence needs. You should have hands-on experience with Python or SQL, Google Cloud Platform (GCP), and a strong understanding of data management, quality, and security best practices. Key Responsibilities: - Build and maintain moderately complex data pipelines to ensure data flow, transformation, and usability for analytical projects. - Design and implement data models, optimizing for performance and scalability. - Develop rules and tracking processes to support data quality models based on data characteristics and supply patterns. - Prepare data for analytical use by gathering, integrating, cleansing, and structuring data from multiple sources and systems. - Troubleshoot pipeline and data issues to maintain accuracy and reliability. - Stay updated with GCP advancements and suggest innovative solutions. - Implement security best practices within data pipelines and cloud infrastructure. - Collaborate with global teams to share and adopt best practices in data management, maintenance, reporting, and security. - Execute data quality checks to ensure consistency and integrity. - Work with credit data products and perform analysis using tools like Google BigQuery, BigTable, DataFlow, and Spark/PySpark. Mandatory Skills: - Proficiency in Python or SQL with experience in scripting for data manipulation and processing. - Intermediate understanding and experience with Google Cloud Platform (GCP) and cloud computing concepts. - Proven ability to build, maintain, and troubleshoot moderately complex data pipelines. - Experience in designing and optimizing data models for performance. - Ability to develop rules, tracking processes, and checks to support a data quality model. - Skilled in integrating, consolidating, cleansing, and structuring data for analytical use. - Knowledge of security best practices in pipelines and cloud infrastructure. - Hands-on experience with Google BigQuery, BigTable, DataFlow, Scala + Spark or PySpark. - Experience working with JSON, AVRO, and PARQUET formats. - Strong communication skills to promote global best practices and guide adoption. Preferred Qualifications: - Cloud certification (e.g., GCP Data Engineer, AWS, Azure). - Experience with credit data products. - Familiarity with data governance frameworks and metadata management tools. Technical Skills: Python, SQL, GCP, BigQuery, BigTable, DataFlow, Spark / PySpark, JSON, AVRO, PARQUET (Note: The company details were not explicitly mentioned in the provided job description.) As a skilled Data Engineer with 6 to 8 years of experience, your role will involve designing, building, and maintaining data pipelines and data models that support analytical and business intelligence needs. You should have hands-on experience with Python or SQL, Google Cloud Platform (GCP), and a strong understanding of data management, quality, and security best practices. Key Responsibilities: - Build and maintain moderately complex data pipelines to ensure data flow, transformation, and usability for analytical projects. - Design and implement data models, optimizing for performance and scalability. - Develop rules and tracking processes to support data quality models based on data characteristics and supply patterns. - Prepare data for analytical use by gathering, integrating, cleansing, and structuring data from multiple sources and systems. - Troubleshoot pipeline and data issues to maintain accuracy and reliability. - Stay updated with GCP advancements and suggest innovative solutions. - Implement security best practices within data pipelines and cloud infrastructure. - Collaborate with global teams to share and adopt best practices in data management, maintenance, reporting, and security. - Execute data quality checks to ensure consistency and integrity. - Work with credit data products and perform analysis using tools like Google BigQuery, BigTable, DataFlow, and Spark/PySpark. Mandatory Skills: - Proficiency in Python or SQL with experience in scripting for data manipulation and processing. - Intermediate understanding and experience with Google Cloud Platform (GCP) and cloud computing concepts. - Proven ability to build, maintain, and troubleshoot moderately complex data pipelines. - Experience in designing and optimizing data models for performance. - Ability to develop rules, tracking processes, and checks to support a data quality model. - Skilled in integrating, consolidating, cleansing, and structuring data for analytical use. - Knowledge of security best practices in pipelines and cloud infrastructure. - Hands-on experience with Google BigQuery, BigTable, DataFlow, Scala + Spark or PySpark. - Experience working with JSON, AVRO, and PARQUET formats. - Strong communication skills to promote global best practices and guide adoption. Preferred Qualific
