Data Engineer

Job Details

Role:

Location:

Experience:

Qualifications for Data Engineer

• 2+ Years of experience in building and optimizing big data’ solutions required to fulfill business and technology requirements.
• 2+ years of technical expertise in areas of design and implementation using big data technology Hadoop, Hive, PySpark, Python/Java.
• Strong analytic skills to understand and create solutions for business use cases.
• Ensure best practices to implement data governance principles, data quality checks on each data layer.
• A successful history of manipulating, processing and extracting value from large disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment.

We are looking for a candidate with 3+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.

They should also have experience using the following software/tool-

• Experience with big data tools: Hadoop, Hive, PySpark, Kafka, Airflow.
• Experience with relational SQL and NoSQL databases: MySQL, Postgres and Cassandra.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with AWS/Azure cloud services: EC2, EMR, RDS, Redshift, HDInsights, DataLake, DataBricks
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with object-oriented/object function scripting languages: Java, Python, Scala, etc.