Lead II - Data Engineering

Role Overview: As a Lead II Data Engineering at UST, you will be responsible for designing, building, and maintaining data pipelines and infrastructure to support the Data Science team. Leveraging GCP tools and technologies, you will handle data ingestion, transformation, and optimization for machine learning applications. Your role will be crucial in ensuring high data quality and collaborating with cross-functional teams to deliver data-driven solutions that drive business innovation. Key Responsibilities: - Architect, build, and maintain scalable, secure data pipelines using GCP tools such as BigQuery, Dataflow, Dataproc, and Pub/Sub. - Write Python and SQL scripts to transform and load data for ML pipelines. - Optimize data access and performance through techniques like partitioning, indexing, and caching. - Ensure data quality and availability for ML/analytics using tools like Data Catalog and Data Studio. - Monitor and troubleshoot data issues utilizing GCP tools like Stackdriver and Cloud Monitoring. - Document pipelines and adhere to best practices using tools like Git and Jupyter. - Collaborate with data scientists, engineers, and stakeholders to understand requirements and deliver data solutions. - Support generative AI initiatives by preparing and enhancing datasets using augmentation, anonymization, and synthesis techniques. Qualification Required: - Strong experience in Python and SQL for data engineering tasks. - Hands-on expertise in Google Cloud Platform (GCP) services, particularly BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Storage. - Experience in building and maintaining data pipelines and ETL/ELT processes. - Knowledge of data modeling, data quality, data governance, and security practices. - Understanding of ML pipeline integration and data preparation for AI/ML models. About UST: UST is a global digital transformation solutions provider with a track record of over 20 years. With over 30,000 employees across 30 countries, UST collaborates with the world's leading companies to drive real impact through transformation. Their approach, powered by technology and led by purpose, embeds innovation and agility into their clients' organizations, touching billions of lives in the process. Role Overview: As a Lead II Data Engineering at UST, you will be responsible for designing, building, and maintaining data pipelines and infrastructure to support the Data Science team. Leveraging GCP tools and technologies, you will handle data ingestion, transformation, and optimization for machine learning applications. Your role will be crucial in ensuring high data quality and collaborating with cross-functional teams to deliver data-driven solutions that drive business innovation. Key Responsibilities: - Architect, build, and maintain scalable, secure data pipelines using GCP tools such as BigQuery, Dataflow, Dataproc, and Pub/Sub. - Write Python and SQL scripts to transform and load data for ML pipelines. - Optimize data access and performance through techniques like partitioning, indexing, and caching. - Ensure data quality and availability for ML/analytics using tools like Data Catalog and Data Studio. - Monitor and troubleshoot data issues utilizing GCP tools like Stackdriver and Cloud Monitoring. - Document pipelines and adhere to best practices using tools like Git and Jupyter. - Collaborate with data scientists, engineers, and stakeholders to understand requirements and deliver data solutions. - Support generative AI initiatives by preparing and enhancing datasets using augmentation, anonymization, and synthesis techniques. Qualification Required: - Strong experience in Python and SQL for data engineering tasks. - Hands-on expertise in Google Cloud Platform (GCP) services, particularly BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Storage. - Experience in building and maintaining data pipelines and ETL/ELT processes. - Knowledge of data modeling, data quality, data governance, and security practices. - Understanding of ML pipeline integration and data preparation for AI/ML models. About UST: UST is a global digital transformation solutions provider with a track record of over 20 years. With over 30,000 employees across 30 countries, UST collaborates with the world's leading companies to drive real impact through transformation. Their approach, powered by technology and led by purpose, embeds innovation and agility into their clients' organizations, touching billions of lives in the process.