Automation Developer

In this role, you will be developing and implementing quantitative software applications to process and analyze real-time financial market data in a high-performance computing environment. Your responsibilities will include maintaining and optimizing existing software applications, as well as recommending and implementing improvements. You will design and develop solutions using Python and PySpark in a spark/Hadoop environment. Additionally, you will build, optimize, and troubleshoot ETL workflows to extract, transform, and load data from diverse sources into big data platforms. Collaborating with data engineers, data scientists, and business stakeholders to gather requirements and deliver data-driven solutions will be a key part of your role. You will ensure data quality, consistency, and integrity throughout the data lifecycle, monitoring, debugging, and optimizing data processing jobs for performance and scalability. Documenting technical solutions, workflows, and best practices for knowledge sharing is also expected. Qualifications and Skills: - Proven experience in automation development with Python and PySpark - Excellent coding skills and ability to write stable, maintainable, and reusable code - Strong knowledge of data manipulation and visualization tools (e.g., Pandas, Matplotlib, Seaborn) - Familiarity with Linux/OS X command line, version control software (git) - Strong understanding of big data ecosystems (e.g., Apache Spark, Hadoop) and distributed computing - Experience with SQL/NoSQL databases such as MySQL and MongoDB - Good understanding of RESTful APIs Good to have skills: - Understanding in Statistics, e.g., hypothesis formulation, hypothesis testing, descriptive analysis, and data exploration - Understanding in Machine Learning, e.g., linear/logistics regression discriminant analysis, boosting/bagging, etc. - Ambition to learn and implement current state-of-the-art machine learning frameworks such as Scikit-Learn, TensorFlow, and Spark Academic Qualifications: - Bachelor or equivalent degree in Computer Science, Computer Engineering, IT,