Data Engineer

Skilled Data Engineer with 4- 6 years of experience; Expertise in Python, REST APIs, data pipeline development, data extraction, transformation, loading(ETL process) and dashboard creation. Strong analytical skillset to handle both structured and non-structured data set. Hands-on experience with Azure, Pyspark, Apache Kafka, apache nifi and working with time series data. Responsible for building robust data solutions that drive business insights and support analytics initiatives.

Responsibilities:

• Design, develop, and maintain scalable data pipelines using Python and Azure services.
• Hands-on data pipeline.
• Integrate and process data from various sources using REST APIs.
• Implement and manage real-time data ingestion and streaming solutions with Apache Kafka.
• Develop interactive dashboards to visualize data and present actionable insights to stakeholders.
• Work extensively with time series data, ensuring data quality, storage, and efficient retrieval.
• Collaborate with other data engineers, analysts, and business users to understand requirements and deliver solutions.
• Optimize data workflows for performance, scalability, and reliability.
• Maintain clear documentation of data flows, pipeline architecture, and dashboard designs.
• Ensure data security and compliance with organizational and industry standards.

Required Skills & Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or related field.
• 4 to 6 years of experience in data engineering or a related role.
• Proficiency in Python for data manipulation and pipeline development.
• Experience with REST API integration for data extraction and processing.
• Strong hands-on experience with Azure data services (such as Azure Data Factory, Azure Databricks, Azure SQL, etc.).
• Solid understanding and practical experience with Apache Kafka for data streaming.
• Demonstrated experience in designing and building dashboards using tools like Power BI, Tableau, or similar platforms.
• Experience working with time series data: storage, processing, and analysis.
• Strong problem-solving skills and attention to detail.
• Excellent communication and teamwork skills.