Software Engineer

As an experienced professional with over 3 years of experience in Big Data & data related technologies, you possess expert level understanding of distributed computing principles. Your expertise lies in Apache Spark and you have hands-on programming skills in Python. You are proficient in Hadoop v2, Map Reduce, HDFS, and Sqoop. Additionally, you have experience in building stream-processing systems using technologies like Apache Storm or Spark-Streaming. Key Responsibilities: - Hands-on experience in Apache Spark - Proficient in programming with Python - Strong knowledge of Hadoop v2, Map Reduce, HDFS, and Sqoop - Building stream-processing systems using Apache Storm or Spark-Streaming - Working with messaging systems such as Kafka or RabbitMQ - Utilizing Big Data querying tools like Hive and Impala - Integrating data from multiple sources including RDBMS, ERP, and Files - Knowledge of SQL queries, joins, stored procedures, and relational schemas - Experience with NoSQL databases like HBase, Cassandra, and MongoDB - Familiarity with ETL techniques and frameworks - Performance tuning of Spark Jobs - Working with native Cloud data services AWS or AZURE Databricks, GCP - Leading a team efficiently - Designing and implementing Big Data solutions - Practicing AGILE methodology Qualifications Required: - 3+ years of experience in Big Data & data related technologies - Expert level understanding of distributed computing principles - Hands-on programming experience with Python - Proficiency in Apache Spark - Experience with various Big Data technologies such as Hadoop, Kafka, Hive, Impala, etc. - Knowledge of SQL, NoSQL databases, ETL techniques, and Cloud data services - Ability to lead a team effectively - Familiarity with AGILE methodology Feel free to omit any additional details of the company if not present in the job description.,