Software Engineer Data

As a highly skilled Software Engineer with strong Python expertise and a solid understanding of data engineering principles, you will join the team to develop and optimize scalable applications and data workflows, integrating diverse data sources, and supporting the development of data-driven products. Your role will involve hands-on experience in software development, data modeling, ETL/ELT pipelines, APIs, and cloud-based data systems. **Key Responsibilities:** - Design, develop, and maintain Python-based applications, APIs, and microservices focusing on performance, scalability, and reliability. - Write clean, modular, and testable code following best software engineering practices. - Participate in code reviews, debugging, and optimization of existing applications. - Integrate third-party APIs and services for application features or data ingestion. - Build and optimize data pipelines (ETL/ELT) for ingesting, transforming, and storing structured and unstructured data. - Work with relational and non-relational databases to ensure efficient query performance and data integrity. - Collaborate with analytics and ML teams to ensure data availability, quality, and accessibility for downstream use cases. - Implement data modeling, schema design, and version control for data pipelines. - Deploy and manage solutions on cloud platforms (AWS/Azure/GCP) using services like S3, Lambda, Glue, BigQuery, or Snowflake. - Implement CI/CD pipelines and participate in DevOps practices for automated testing and deployment. - Monitor and optimize application and data pipeline performance using observability tools. - Work cross-functionally with software engineers, data scientists, analysts, and product managers to understand requirements and translate them into technical solutions. - Provide technical guidance and mentorship to junior developers and data engineers. - Document architecture, code, and processes for maintainability and knowledge sharing. **Qualification Required:** - Hands-on experience with Databricks environment. - Bachelors/Masters degree in Computer Science, Engineering, or a related field. - 3+ years of experience in Python software development. - Strong knowledge of data structures, algorithms, and object-oriented programming. - Hands-on experience in building data pipelines using Airflow, Luigi, Prefect, or custom ETL frameworks. - Proficiency with SQL and database systems (PostgreSQL, MySQL, MongoDB, etc.). - Experience with cloud services (AWS/GCP/Azure) and containerization (Docker, Kubernetes). - Familiarity with message queues/streaming platforms (Kafka, Kinesis, RabbitMQ) is a plus. - Strong understanding of APIs, RESTful services, and microservice architectures. - Knowledge of CI/CD pipelines, Git, and testing frameworks like PyTest, UnitTest. (Note: Additional details about the company were not included in the provided job description.),