7.0 - 11.0 years

0 Lacs

pune, maharashtra

On-site

As a Lead Data Engineer specializing in Databricks, your role involves designing, developing, and optimizing scalable, efficient, and secure data solutions. Your primary responsibilities include: - Leading the design, development, and optimization of data solutions using Databricks. - Collaborating with cross-functional teams to gather and analyze data requirements, translating them into robust data architectures and solutions. - Developing and maintaining ETL pipelines, leveraging Databricks and integrating with Azure Data Factory as needed. - Implementing machine learning models and advanced analytics solutions, incorporating Generative AI to drive innovation. - Ensuring data quality, governance, and security practices are adhered to, maintaining the integrity and reliability of data solutions. - Providing technical leadership and mentorship to junior engineers, fostering an environment of learning and growth. - Staying updated on the latest trends and advancements in data engineering, Databricks, Generative AI, and Azure Data Factory to continually enhance team capabilities. In order to excel in this role, you are required to have: - A Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. - 7+ to 10 years of experience in data engineering, with a focus on Databricks. - Proven expertise in building and optimizing data solutions using Databricks and integrating with Azure Data Factory/AWS Glue. - Proficiency in SQL and programming languages such as Python or Scala. - Strong understanding of data modeling, ETL processes, and Data Warehousing/Data Lakehouse concepts. - Familiarity with cloud platforms, particularly Azure, and containerization technologies such as Docker. - Excellent analytical, problem-solving, and communication skills. - Demonstrated leadership ability with experience mentoring and guiding junior team members. Additionally, the following skills would be advantageous for this role: - Experience with Generative AI technologies and their applications. - Familiarity with other cloud platforms, such as AWS or GCP. - Knowledge of data governance frameworks and tools. This role offers you the opportunity to lead and innovate in the realm of data engineering, leveraging cutting-edge technologies and fostering a culture of continuous learning and growth within the team.,

Posted 1 day ago

Apply

Senior Software Engineer Intuit

4.0 - 8.0 years

15 - 20 Lacs

bengaluru

Work from Office

Overview Join Intuits Business Intelligence (BI) Platform team as we reimagine the next generation of scalable, intelligent data infrastructure We serve over 240TB of data, 2 billion records daily, and deliver 200+ million report requests through 20+ complex pipelines?supporting enterprise and mid-market customers on their most critical decisions, We are seeking a Senior Data Engineer to join our Data Platform team, with a focus on designing robust data models, building scalable ETL/ELT pipelines, and enabling trustworthy, high-quality data for analytics, reporting, and intelligent systems In this role, you will play a critical part in evolving our data architecture, ensuring data quality, and building integrations that power analytics and decision-making across the business, What you'll bring 6+ years of hands-on experience in data engineering or data platform development, Strong experience in building and optimizing data pipelines using Spark and Flink, Proficiency with DBT for transformation workflows and Kafka for event-driven ingestion, Solid understanding of data modeling principles and best practices in relational and analytical systems, Proven track record in creating and maintaining historical, delta, and snapshot data structures, Familiarity with data quality frameworks and tools for validation and anomaly detection, Experience working with columnar file formats and scalable data storage systems, Strong coding skills in Python or Scala, and familiarity with SQL at scale, Bachelors or Masters degree in Computer Science, Data Engineering, or a related field, How you will lead Design and implement scalable ETL and ELT pipelines using tools like Apache Spark, DBT, and Kafka, Own the development of data models that support reporting, analytics, and machine learning use cases, Build and maintain historical, delta, and snapshot tables optimized for large-scale data processing and access patterns, Work with columnar storage formats (e-g , Parquet, ORC) to optimize performance and storage efficiency, Integrate and automate data validation and quality checks, ensuring trust and accuracy across pipelines, Partner with data platform and product teams to design and deliver seamless data integrations across systems and domains, Contribute to data governance practices, schema evolution, and performance tuning,

Posted 1 week ago

Apply

Azure Databricks DigiHelic Solutions Pvt. Ltd.

5.0 - 9.0 years

0 Lacs

noida, uttar pradesh

On-site

As a Data Engineer with 5 years of experience, you will be based in Noida and work in a hybrid mode. Your responsibilities will include working with AWS/Azure, ETL processes, SQL, Databricks, Python/Scala, Py-Spark/Spark, Apache Nifi, and Hive.,

Posted 1 week ago

Apply

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.