Senior Data Engineer

10 - 14 years

0.6 - 1.0 Lacs P.A.

Pune

Posted:2 months ago| Platform: Naukri logo

Apply Now

Skills Required

Etl PipelinesData IngestionData WarehousingPython DevelopmentData BricksData Transformation

Work Mode

Hybrid

Job Type

Full Time

Job Description

Overview of the role: Guidepoint seeks an experienced Data Engineer as an integral member of the Pune team. The Pune Technology Hub will serve as the base of our new Data/AI team, dedicated to building a modern data infrastructure for advanced analytics and the development of responsible AI. This strategic investment is integral to Guidepoints vision for the future, aiming to develop cutting-edge Generative AI and analytical capabilities that will underpin Guidepoints Next-Gen research enablement platform and data products. This role demands proven technical prowess with Lakehouse architecture to drive the development of next-generation research enablement platforms and advanced data products leveraging AI/ML technologies. The Senior Data Engineer will implement Guidepoints cloud data infrastructure that will advance BI, Analytics and AI/ML use cases across the organization. More about Guidepoint’s Technology Team: Guidepoint’s Technology team thrives on problem-solving and creating happier users. As Guidepoint works to achieve its mission of making individuals, businesses, and the world smarter through personalized knowledge-sharing solutions, the engineering team is taking on challenges to improve our internal application architecture and create new AI-enabled products to optimize the seamless delivery of our services. Your Day-to-Day Will Include : Design and development of systems for the maintenance of the Azure Databricks, ETL processes, business intelligence and data ingestion pipelines for AI/ML use cases. Experience in Knowledge Graph Development and experience building Entity relationship and graph-based knowledge models and experience in NLP based Insights. Create and support ETL pipelines and tables schemas in order to facilitate the accommodation of new and existing data sources for the Lakehouse on Databricks. Maintain data governance and data privacy standards. Collaborate with data architects, data scientists, analysts and other business consumers to quickly and thoroughly analyze business requirements to populate the data warehouse, optimized for reporting and analytics. Perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. Maintain technical documentation and mentor junior data engineers on best practices in data engineering and Lakehouse architecture. Drive innovation and contribute to the development of cutting-edge Generative AI and analytical capabilities for Guidepoint's Next-Gen research enablement platform. Minimum Qualifications: 10+ years of related experience with a bachelor’s degree Proven leadership skills, a strategic mindset, and a passion for driving innovation in data engineering and modern data architecture. Advanced working SQL knowledge and experience working with relational and NoSQL databases, query authoring (SQL) as well as working familiarity with a variety of databases (e.g. SQL Server, Neo4J, Elastic, Cosmos). Experience building and optimizing data pipelines on Azure Databricks. In depth knowledge and hands-on experience with Data Engineering, Data Warehousing and Delta Lake on Databricks. Highly proficient in Spark, Python, SQL Working knowledge of Fivetran is a bonus. A successful history of manipulating, processing and extracting value from large, disconnected datasets. Exceptional stakeholder management and communication skills to effectively communicate across global teams. Preferred Qualifications: Knowledge of BI Tools like PowerBI etc. Experience building and deploying ML and feature engineering pipelines to production using MLFlow. Experience with building data pipeline from various business applications like Salesforce, NetSuite, etc. Knowledge of message queuing, stream processing, and highly scalable data stores. Experience or knowledge of working in a compliance-based environment, including building and deploying compliant software solutions throughout the software life cycle is a nice to have. Familiarity with cloud-based AI/ML services and Generative AI tools.

Information Services
New York NY +30

RecommendedJobs for You

Chennai, Pune, Mumbai, Bengaluru, Gurgaon

Chennai, Pune, Delhi, Mumbai, Bengaluru, Hyderabad, Kolkata

Pune, Bengaluru, Mumbai (All Areas)