Senior Software Engineer (Data Engineering)

As a skilled Data Engineer at BigThinkCode Technologies, your role involves designing, building, and maintaining robust data pipelines and infrastructure to optimize data flow and ensure scalability. Your technical expertise in Python, SQL, ETL/ELT frameworks, and cloud data warehouses, along with strong collaboration skills, will be crucial in partnering with cross-functional teams to enable seamless access to structured and unstructured data across the organization. **Key Responsibilities:** - Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data processing. - Optimize and manage SQL queries for performance and efficiency in handling large-scale datasets. - Collaborate with data scientists, analysts, and business stakeholders to translate requirements into technical solutions. - Ensure data quality, governance, and security across pipelines and storage systems. - Document architectures, processes, and workflows for clarity and reproducibility. **Required Technical Skills:** - Proficiency in Python for scripting, automation, and pipeline development. - Expertise in SQL for complex queries, optimization, and database design. - Hands-on experience with ETL/ELT tools such as Apache Airflow, dbt, and AWS Glue. - Experience working with both structured (RDBMS) and unstructured data (JSON, Parquet, Avro). - Familiarity with cloud-based data warehouses like Redshift, BigQuery, and Snowflake. - Knowledge of version control systems like Git and CI/CD practices. **Preferred Qualifications:** - Experience with streaming data technologies such as Kafka, Kinesis, and Spark Streaming. - Exposure to cloud platforms like AWS, GCP, Azure, and their data services. - Understanding of data modeling techniques like dimensional and star schema and optimization. The company offers benefits like a flexible schedule, health insurance, paid time off, and a performance bonus. Thank you for considering this opportunity at BigThinkCode Technologies.,