Senior Data Engineer

Job Requirements

Skilled

Data Engineer

with 5- 8 years of experience; Expertise in Python, REST APIs, data pipeline development, data extraction, transformation, loading(ETL process) and dashboard creation . Strong analytical skillset to handle both structured and non-structured data set. Hands-on experience with Python, Pyspark, Athena,, strong SQL ,AWS services like Glue ,Lambda and working with any ETL tool. Responsible for building robust data solutions that drive business insights and support analytics initiatives.

Responsibilities

Work Experience

• Design, develop, and maintain scalable data pipelines using Python and AWS services.
• Hands-on data integration with multiple source systems with structured ,semistructered and unstructured data sets.
• Integrate and process data from various sources using REST APIs.
• Implement and manage real-time data ingestion.
• Develop interactive dashboards to visualize data and present actionable insights to stakeholders.
• Work extensively with time series data, ensuring data quality, storage, and efficient retrieval.
• Collaborate with other data engineers, analysts, and business users to understand requirements and deliver solutions.
• Optimize data workflows for performance, scalability, and reliability.
• Maintain clear documentation of data flows, pipeline architecture, and dashboard designs.
• Ensure data security and compliance with organizational and industry standards.
  

Required Skills & Qualifications

• Bachelor’s or Master’s degree , Engineering, Information Systems, or related field.
• 5 to 8 years of experience in data engineering or a related role.
• Proficiency in Python for data manipulation and pipeline development.
• Experience with REST API integration for data extraction and processing.
• Strong hands-on experience with Azure data services (such as Azure Data Factory, Azure Databricks, Azure SQL, etc.).
• Solid understanding and practical experience with Apache Kafka for data streaming.
• Demonstrated experience in designing and building dashboards using tools like Power BI, Tableau, or similar platforms.
• Experience working with time series data: storage, processing, and analysis.
• Strong problem-solving skills and attention to detail.
• Excellent communication and teamwork skills.