Senior Data Engineer

As a member of the Natural Language Query (NLQ) platform team, you will be responsible for providing Platform-as-a-Service offerings that are easy to consume, scalable, secure, and reliable using open source-based solutions. You will also play a key role in developing and enhancing the NLQ Copilot platform to meet the evolving needs of multi-tenant environments. Your responsibilities will include implementing Context builder algorithms to generate 100% accurate SQL based on customer needs and collaborating with downstream clients to integrate business requirements. Additionally, you will work closely with data scientists, engineers, and product managers to optimize the performance of the NLQ platform. Utilizing cutting-edge NLP/LLM and machine learning techniques will be essential to improve the accuracy and efficiency of query transformations. Ensuring scalability and reliability through rigorous testing and continuous improvement will also be part of your role. Championing the adoption of open infrastructure solutions and spending 80% of your time writing code in different languages, frameworks, and technology stacks are essential aspects of this position. Qualifications: Basic Qualifications: - 4+ years of experience in architecture design and development of large-scale data management platforms with simple solutions - Bachelor's or master's degree in Computer Science or related technical discipline - Extensive hands-on coding and designing skills in Java/Python for backend development - Proficiency in MVC (model-view-controller) for end-to-end development - Familiarity with SQL/NoSQL technology and databases such as Oracle, DB2, SQL Server - Experience with Web Services (REST/SOAP/gRPC) - Knowledge of React/Angular for front-end development (UI front-end is a plus) - Expertise in designing and managing complex data structures and processes - Efficiently leveraging distributed big data systems like Hadoop Hive, Spark, Kafka streaming - Strong service architecture and development experience focusing on high performance and scalability - Strong leadership skills and ability to work effectively in a team - Mentoring and growing junior team members Preferred Qualifications: - Deep knowledge and hands-on experience with big data and cloud computing technologies - Experience with LLM/GenAI tools/applications and Prompt Engineering - Familiarity with ETL/ELT tools/applications, Apache NiFi, and Apache Spark for processing large datasets - Knowledge of Elastic Search, Data Catalog tools, Data Pipeline development tools - Experience with Data Governance and Data Quality tools,