Job
Description
Purpose Key Responsibility Perform IT system architecture design, new technology research, and provide recommendation. Design and implement optimal data pipeline architecture (considered high data volume, data governance, etc.). Work with PRODUCT/BIZ teams to assist with new data platform re-engineering or data-related technical issues. Build and implement the features and services of an intelligent device control platform on Kubernetes (K8S) environment Build multiple data connectivity capability Perform cloud native microservices design and build service library for workflow pipeline integration. Design solution architecture for non-stop services upgrade and operation Implement service message exchange with new technology Build and implement the features and services of Big Data analytic platform on high availability NoSQL DB (e.g.Cassandra, HDFS, S3/MinIO, MariaDB, MongoDB etc.) on K8s environment. Build small data lake initially then enlarge for company data lake Build small image/video lake initially then enlarge Qualification B. E in Computer Science, Electrical Engineering, Mechanical Engineering or related fields (Information Engineering, Information Management, Data engineering, Applied Mathematics, Robot/Automation. Experience Experience in container and cloud-native solution (K8S/Docker, Istio, ArgoCD, MongoDB, Redis, …etc) Experience in microservice implementation, conductor workflow implementation, Kafka queue implementation on K8S environment Experience in tool connection to IT systems (IoT, Image/Voice, SECS/GEM, PLC/DCS, add-on sensors... etc) Experience in big data ecosystem (Hadoop, Spark, Cassendra...etc) and SQL NoSQL database (MariaDB, MongoDB, MinIO(S3)...etc). Experience in data orchestration among various data sources, data parallel processing, metadata data management, data quality assurance, data security and governance, and new data technology evaluation and enablement. Knowledge of object stores for image and semi/unstructured data. Experience in big data solutions development, data pipelines for batch and streaming applications. Experience in monitoring and analysis tools, such as Nagios, ELK, Splunk, and Grafana. Ability to design solutions independently based on high-level architecture. Cloud native deployment practices, specifically, containers and container orchestration platforms. Additional Technical Requirements Experience in Hadoop ecosystem and SQL NoSQL databases. Experience in TypeScript/JavaScript, Kotlin, Java, F#, C/C++/C#, Rust, SQL or other relevant programming languages Big Data Solutions development, data pipelines for batch and streaming applications. Competencies Teamwork Agility Customer Empathy Purposeful Pursuit of Goals Pioneering Deep Domain Knowledge Future Ready Show more Show less
