Data Engineer (Python + AWS)

As a Data Engineer at our company, you will play a crucial role in designing, building, and managing robust and scalable data pipelines using Python, SQL, and AWS. Your key responsibilities will include: - Designing, building, and managing robust and scalable data pipelines (batch and real-time). - Developing ETL processes to acquire, transform, and integrate data from multiple sources. - Building and maintaining data warehouses, data lakes, and other storage solutions. - Optimizing data systems for performance, reliability, and scalability. - Collaborating with cross-functional teams to understand data requirements and deliver efficient solutions. - Ensuring data quality, consistency, and integrity across all systems and pipelines. - Implementing data governance, privacy, and security best practices. - Monitoring and troubleshooting data flows and pipelines, ensuring high availability and minimal downtime. - Documenting data architecture, workflows, and system designs for maintainability and scalability. **Required Technical Skills:** **Must Have Skills:** - Strong proficiency in Python programming and SQL. - Hands-on experience in ETL design and development using tools such as AWS Glue, Apache Airflow, Luigi, or Talend. - Expertise in AWS Cloud Services (S3, Redshift, Lambda, Glue). - Experience with Big Data frameworks like Apache Spark, Hadoop, or Kafka. - Strong understanding of data modeling, data warehousing, and data architecture principles. - Proficiency in Git version control and CI/CD automation pipelines. **Good to Have Skills:** - Exposure to GCP or Azure cloud platforms. - Familiarity with data orchestration tools and data governance frameworks. - Experience working with structured and unstructured data at scale. In addition to technical skills, you are expected to demonstrate the following professional attributes: - Excellent problem-solving and analytical thinking skills. - Strong communication and collaboration across teams. - Proven ability to deliver end-to-end data solutions with minimal supervision. - Detail-oriented mindset with a passion for data quality and automation. **Educational Qualification:** - Bachelors or Masters degree in Computer Science, Engineering, Information Systems, or a related field. - Minimum 15 years of full-time education required.,