Data Engineer

Experience:

Job Location: Remote

No. of Position:

Qualifications:

Work Timings:

Functional Area:

We are seeking a skilled Data Engineer with 4 to 8 years of experience to join our team. The ideal candidate will have a strong background in Python programming, along with expertise in AWS or Azure services. The candidate should also possess solid SQL skills and be proficient in web scraping techniques.

Role and responsibilities:

• Develop and maintain data pipelines using Python, PySpark, and SQL to extract, transform, and load data from various sources.
• Implement and optimize data processing workflows on AWS or Azure cloud platforms.
• Utilize Databricks or Azure data factory for efficient data storage and processing.
• Develop and maintain web scraping scripts to gather data from online sources.
• Collaborate with cross-functional teams to design and implement API endpoints for data access.
• Work on UI Path automation projects to streamline data extraction and processing tasks.
• Develop and maintain Django or Flask web applications for internal data management and visualization.
• Leverage Pandas and other data manipulation libraries for data analysis and preprocessing.
• Enhance API development skills for integrating data services with external systems.
• Stay updated with the latest industry trends and technologies, such as Flask, PyTorch, etc., to continuously improve data engineering processes.

Skills, Knowledge, Experience:

• Bachelor's degree in Computer Science, Engineering, or related field.
• 4 to 8 years of experience in data engineering roles.
• Proficiency in Python programming language.
• Strong understanding of AWS or Azure cloud services.
• Solid SQL skills for querying and manipulating data.
• Previous experience with web scraping techniques and tools.
• Hands-on experience with Django web framework.
• Knowledge of API development and integration.
• Experience with PySpark for big data processing.
• Proficiency in Pandas for data manipulation and analysis.
• Familiarity with UI Path for automation or Power Automate is advantageous.
• Experience with Databricks.
• Familiarity with Flask and PyTorch is a plus.
• Experience working with USA or European clients is a plus.
• Experience working with multi-vendor, multi-culture, distributed offshore, and onshore development teams in a dynamic and complex environment will be helpful in day-to-day working.
• Must have excellent written and verbal communication skills. The candidate should be able to present his suggestions and explain the technical approach.