Big Data Engineer

Role Overview: As a Senior Data Engineer at the company, your primary responsibility will be designing, developing, and maintaining scalable data pipelines and creating new API integrations to handle the increasing data volume and complexity. You will work closely with analytics and business teams to enhance data models that drive business intelligence tools, facilitating data-driven decision-making across the organization. Key Responsibilities: - Designing, constructing, testing, and maintaining highly scalable data management systems and data pipelines. - Ensuring alignment of systems with business requirements and industry standards. - Developing high-performance algorithms, prototypes, predictive models, and proof of concepts. - Researching data acquisition opportunities and exploring new applications for existing data. - Creating data set processes for data modeling, mining, and production. - Integrating new data management technologies and software engineering tools into the current infrastructure. - Building custom software components and analytics applications. - Installing and updating disaster recovery procedures. - Collaborating with data architects, modelers, and IT team members to achieve project objectives. - Providing technical consulting to peer data engineers during the design and development of complex data projects. Qualification Required: - A Bachelor's degree in computer science, engineering, or a related field, or equivalent work experience. - 5-8 years of proven experience as a Senior Data Engineer or in a similar role. - Proficiency with big data tools such as Hadoop, Spark, Kafka, Ansible, Chef, Terraform, Airflow, and Protobuf RPC. - Expertise in SQL for data manipulation (DML) and validation (DB2). - Experience with data pipeline and workflow management tools. - Familiarity with object-oriented/object function scripting languages such as Python, Java, and Go. - Strong problem-solving and analytical skills. - Excellent verbal and interpersonal communication skills. - Ability to provide technical leadership within the team.,