Associate Software Engineer

Job Description

What you will do In this vital role We are seeking a motivated and detail-oriented Associate Software Engineer to join our Data Catalog & Governance team within the pharmaceutical domain. This role supports the development, integration, and maintenance of systems that enable data discovery, metadata management, and regulatory compliance across a modern data platform. You will work complex engineering problems across distributed software development, optimizing data access and delivery, enabling core access controls via well-defined security paradigms, building UIs to enable data visualization, using machine learning to curate data, or engaging with businesses to ensure their data needs are met Assist in building APIs, connectors, and automation scripts to ingest metadata and support lineage tracking. Assist in the development and maintenance of data platform services and applications using modern programming languages (e.g., Python, Java, Scala). Support the design and implementation of data pipelines and APIs to ingest, process, and expose structured and unstructured data. Design & develop modern data management tools to curate our most important data sets, models and processes, while identifying areas for process automation and further efficiencies Develop backend processes to support automated metadata ingestion, data profiling, classification, and tagging Work with cloud-based data technologies and platforms (e.g., AWS, Azure, GCP, Snowflake, Databricks). Contribute to deploying and maintaining platform components using infrastructure-as-code and containerization tools (e.g., Docker, Kubernetes, Terraform). Keeps abreast of current technology trends and familiarize yourself with application development best practices. Demonstrate strong initiative and ability to work with minimal supervision or direction Strong experience with cloud infrastructure (AWS, Azure, or GCP) and infrastructure as code like Terraform In-depth knowledge of relational and columnar SQL databases, including database design Expertise in data warehousing concepts (e.g. star schema, entitlement implementations, SQL v/s NoSQL modeling, milestone, indexing, partitioning) Experience with distributed, multi-tiered systems, algorithms, and relational databases. Possesses strong rapid prototyping skills and can quickly translate concepts into working code Develop and execute unit tests, integration tests, and other testing strategies to ensure the quality of the software Analyze and understand the functional and technical requirements of applications Identify and resolve software bugs and performance issues Work closely with multi-functional teams, including product management, design, and QA, to deliver high-quality software on time Maintain detailed documentation of software designs, code, and development processes Customize modules to meet specific business requirements Work on integrating with other systems and platforms to ensure seamless data flow and functionality Provide ongoing support and maintenance for applications, ensuring that they operate smoothly and efficiently. Informing various stakeholders about any problems with the current technical solutions being implemented. Take ownership and accountability of the deliverables Basic Qualifications: Degree in computer science & engineering preferred with 2-5 years of software development experience Proficient in Python, Java, CSS HTML Hands on experience with various cloud services, understand pros and cons of various cloud services in well architected cloud design principles Proficient in programming languages, Java, Python, Fast Python, Databricks/RDS, Data engineering, S3Buckets, ETL, Hadoop, Spark, airflow, AWS Lambda Working knowledge of open-source tools such as AWS lambda Work with cloud-based data technologies and platforms (e.g., AWS, Azure, GCP, Snowflake, Databricks) Contribute to deploying and maintaining platform components using infrastructure-as-code and containerization tools (e.g., Docker, Kubernetes, Terraform) Strong problem solving, analytical skills; Ability to learn quickly; Excellent communication and interpersonal skills Preferred Qualifications: Experience in Python or Java or Scala is desirable Strong knowledge of data modelling, ETL processes, and database technologies Experience with big data processing frameworks (Hadoop, Spark) and data pipeline orchestration tools (Airflow). Experienced with API integration, serverless, microservices architecture. Experienced with Python, Fast Pythons API Experience in Data bricks, PySpark, Spark, SQL, ETL, ELT Solid understanding of data governance, data security, and data quality best practices Experience with Unit Testing, Building and Debugging the Code Experienced with AWS /Azure Platform, Building and deploying the code Experience in vector database for large language models, Databricks or RDS Experience with DevOps CICD build and deployment pipeline Experience in Agile software development methodologies Experience in End-to-End testing Experience in additional Modern Database terminologies. Good to Have Skills Willingness to work on AI Applications Experience with popular large language models Experience with LangChain or LlamaIndex framework for language models Experience with prompt engineering, model fine tuning Knowledge of NLP techniques for text analysis and sentiment analysis Soft Skills: Excellent analytical and troubleshooting skills. Strong verbal and written communication skills. Ability to work effectively with global teams. High degree of initiative and self-motivation. Team-oriented, with a focus on achieving team goals. Strong presentation and public speaking skills.