Lead II - Data Engineering

Role Overview: As a Lead II Data Engineering at UST, you will design, build, and maintain data pipelines and infrastructure to support the Data Science team. Leveraging GCP tools and technologies, you'll handle data ingestion, transformation, and optimization for machine learning (ML) applications. Your role will involve ensuring high data quality and collaborating with cross-functional teams to deliver data-driven solutions that drive business innovation. Key Responsibilities: - Architect, build, and maintain scalable, secure data pipelines using GCP tools such as BigQuery, Dataflow, Dataproc, and Pub/Sub. - Write Python and SQL scripts to transform and load data for ML pipelines. - Optimize data access and performance using techniques like partitioning, indexing, and caching. - Ensure data quality and availability for ML/analytics using tools like Data Catalog and Data Studio. - Monitor and troubleshoot data issues using GCP tools like Stackdriver and Cloud Monitoring. - Document pipelines and follow best practices using Git, Jupyter, etc. - Collaborate with data scientists, engineers, and stakeholders to understand requirements and deliver data solutions. - Support generative AI initiatives by preparing and enhancing datasets using augmentation, anonymization, and synthesis techniques. Qualification Required: - Strong experience in Python and SQL for data engineering tasks. - Hands-on expertise in Google Cloud Platform (GCP) services, particularly BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Storage. - Experience building and maintaining data pipelines and ETL/ELT processes. - Knowledge of data modeling, data quality, data governance, and security practices. - Understanding of ML pipeline integration and data preparation for AI/ML models. Additional Company Details: UST is a global digital transformation solutions provider known for collaborating with the world's best companies to make a real impact through transformation. With over 30,000 employees in 30 countries, UST partners with clients from design to operation, embedding innovation and agility into organizations for boundless impact.,