DATA ENGINEER – with BI

Experience : 3-4 yrs

Location Hyderabad (Hybrid)

Share your cv to

snehal.patil@p99soft.com

Key Responsibilities: -

• Design, build, and sustain efficient, scalable, and performant Data Engineering Pipelines to ingest, sanitize, transform (ETL/ELT), and deliver high-volume, high-velocity data from diverse sources.
• Ensure reliable and consistent processing of versatile workloads of granularity such as Real Time, Near Real Time, Mini-batch, Batch, and On-demand.
• Translate business requirements into technical specifications, design and implement solutions with clear documentation.
• Develop, optimize, and support production data workflows to ensure comprehensive and accurate datasets for analytics and BI reporting.
• Apply a Software Engineering mindset/discipline, adopting best practices in writing modular code that is maintainable, scalable, and performant.
• Use orchestration and scheduling tools (e.g., Airflow, GitLab Runners) to streamline workflows.
• Automate deployment and monitoring of data workflows using CI/CD best practices. Use DevOps best practices to instrument retries and self-healing to minimize, if not avoid, manual intervention.
• Use AI/Gen AI tools and technologies to build/generate reusable code.
• Collaborate with Engineering Groups, Analysts, Product/Program Management, and other stakeholders to deliver end-to-end solutions.
• Promote strategies to improve our data modelling, quality, architecture, and BI capabilities.
• Mentor junior engineers and contribute to team knowledge sharing.
• Have an analytics and BI mindset to explore and identify opportunities for improved metrics, insights, and dashboards contributing to business growth.
  

Qualifications

• Master’s or bachelor’s degree in computer science or associated discipline with relevant industry experience in a data engineering role.
• Strong proficiency in writing and analysing complex SQL, Python, or any 4GL.
• Strong hands-on experience with Cloud Data Warehouses like Snowflake, Redshift, Big Query.
• Expertise & experience with open table formats (e.g., Apache Iceberg).
• Expertise & experience with distributed data processing frameworks (e.g., Apache Spark, Flink, Beam, Trino).
• Experience with real-time/streaming data technologies (e.g., Kafka, Kinesis, Spark Streaming).
• Experience in using Airflow.
• Deep knowledge in data modelling, performance optimization techniques, and BI best practices.
• Experience in version control and CI/CD workflows (e.g., Git).
• Strong familiarity with BI tools such as Looker, Tableau, Power BI, dashboard design
• Experience working in Agile development environments and with tools like JIRA or Confluence.
• Strong problem-solving, analytical, and debugging skills.
• Excellent communication and collaboration skills, with the ability to work across business, analytics, BI, and engineering teams.