Back-End Developer AI-Driven Data Quality Assessment

Role Overview: As a Back-End Developer at SuperZoom, CBRE's data quality platform, your main responsibility will be to build new capabilities by focusing on developing a backend pipeline. This pipeline will be utilized to process client asset files, run data profiling and quality checks, and integrate LLM-based assessments via API calls such as ChatGPT. Your expertise in data processing, API integrations, and scalable backend architecture will play a crucial role in achieving success in this position. Key Responsibilities: - Develop a backend pipeline to ingest, profile, and assess client asset data files. - Implement data profiling and quality checks using appropriate libraries and frameworks. - Integrate LLM-based assessments by calling APIs like OpenAI, Azure AI. - Optimize processing for large datasets with efficient workflows. - Collaborate with front-end developers, data quality teams, and business stakeholders. - Ensure security and compliance in handling client data. - Maintain scalability and performance in cloud or on-prem infrastructure. Qualification Required: - 5+ years of backend development experience, focusing on data processing and API integration. - Strong Python skills, with experience in Pandas, PySpark, or similar libraries for handling large datasets. - Proficiency in SQL and experience with PostgreSQL, Snowflake, or MS SQL Server. - Experience with integrating LLM APIs for text-based analysis. - Hands-on experience with RESTful APIs and authentication methods like OAuth and API keys. - Strong understanding of data quality and profiling techniques. - Knowledge of Docker and containerized deployments. Additional Details: The company emphasizes the following skills as nice-to-have: - Experience with Apache Airflow for workflow orchestration. - Exposure to cloud platforms (AWS, Azure, GCP) for scalable backend development. - Experience with vector databases and embedding-based retrieval. - Familiarity with LLM fine-tuning or prompt engineering. Please note that the success metrics for this role include: - Successful implementation of automated data profiling and LLM-based assessment pipeline. - Scalability and efficiency in processing large client data files. - Accurate and meaningful AI-driven assessments delivered via API calls. - On-time delivery of backend features in coordination with the front-end team.,