Software Engineer - Data Engineering

As a Data Engineer at our company, you will have the following responsibilities: - Review and understand business requirements to ensure development tasks are completed within the provided timeline and are thoroughly tested with minimal defects. - Collaborate with a software development team to implement best practices and optimize the performance of Data applications. - Engage with various teams across the company and interact with customers to define, design, and showcase new concepts and solutions. - Conduct research on new Big Data technologies, evaluating their maturity and alignment with business and technology strategies. - Work closely with other developers to meet client needs efficiently. - Operate in a rapid and agile development process to accelerate speed to market while maintaining appropriate controls. - Implement high development and testing standards to uphold the quality of deliverables. - Quickly grasp and translate clients" business challenges into solution-oriented discussions. Qualifications required for this role: - Minimum 4+ years of experience in designing and developing using Data Engineering technology stack and programming languages. - Mandatory experience in Python/Scala/Java, Hadoop, HDFS, MR, Spark SQL, Dataframes, RDDs, SQL, and Hive/Snowflake/SQL Server/Bigquery. - Strong knowledge of the current technology landscape and the ability to foresee technology and industry trends. - Familiarity with Big Data Integration with Third-party or in-house Metadata Management, Data Quality, and Master Data Management Solutions. - Active involvement in the community through articles, blogs, and speaking engagements at conferences. Good to have skills: - Experience with Spark Streaming, Spark ML, Kafka/Flume, Apache NiFi, Apache Airflow/Oozie, and Cloud-Based Data Platforms. - Familiarity with NoSQL Databases like HBase, Cassandra, Neo4j, MongoDB, and Elastic Search.,