Data Engineer

KEY RESPONSIBILITIES Proactively drive data science engineering projects forward with a self-motivated and go-getter attitude, effectively navigating ambiguity and managing at-times incomplete requirements. The Python data/ML engineer role is a hands-on software development role specializing in the development from scratch and delivery of data and machine learning based applications and POCs including generative AI based applications. The core emphasis of the role is on backend engineering and API development, with additional emphasis on data ingestion and data processing pipelines. Build on top of and utilize Azure Cloud or AWS platforms, leveraging familiarity with components such as IAM, storage, compute, services, and application development. Develop, maintain, and optimize Python code bases to ensure performance, readability, and adherence to code standards like PEP8, including implementing comprehensive test coverage. Design, develop, and deploy scalable and performant Python web services and APIs for diverse architectures, including synchronous and asynchronous REST APIs. Implement and maintain event-driven product architectures and batch processing systems to support scalable and efficient data processing. Develop and deploy LLM-based and GenAI applications using tools and frameworks such as OpenAI, HuggingFace Transformers, LlamaIndex, and vector stores/databases like Chroma, FAISS, Qdrant, and Weaviate Utilize the Elastic stack (Elastic, Logstash, Kibana) and Databricks for data processing and analytics as preferred Effectively use version control, containerization, CI/CD pipelines, and the deployment of applications on Azure using Git and Docker. Manage SQL databases, such as PostgreSQL, ensuring efficient backend operations and data integrity. Collaborate effectively with cross-functional teams, including data scientists, engineers, and architects, to build and release data and AI applications. Communicate clearly and effectively in English, facilitating interactions and collaboration within a globally distributed and diverse data science and engineering team.