1265 Azure Databricks Jobs - Page 40

Hiring Azure Data Engineer Azure DE - Azure, Azure data factory, databricks, scala 6-10 yrs Pune, Mumbai, Nagpur, Goa, Noida, Gurgaon, Ahmedabad, Jaipur, Indore, Kolkata, Kochi, Hyderabad, Bangalore, Chennai

Role & responsibilities As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements. Your typical day will involve collaborating with team members to develop innovative solutions and ensure seamless application functionality. Key Responsibilities: ¢ Work on client projects to deliver AWS, PySpark, Databricks based Data engineering & Analytics solutions. €¢ Build and operate very large data warehouses or data lakes. €¢ ETL optimization, designing, coding, & tuning big data processes using Apache Spark. €¢ Build data pipelines & applications to stream and process datasets at low latencies. €¢ Show efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data. Technical Experience: €¢ Minimum of 5 years of experience in Databricks engineering solutions on AWS Cloud platforms using PySpark, Databricks SQL, Data pipelines using Delta Lake.€¢ Minimum of 5 years of experience years of experience in ETL, Big Data/Hadoop and data warehouse architecture & delivery. €¢ Minimum of 2 years of experience years in real time streaming using Kafka/Kinesis€¢ Minimum 4 year of Experience in one or more programming languages Python, Java, Scala.€¢ Experience using airflow for the data pipelines in min 1 project.€¢ 1 years of experience developing CICD pipelines using GIT, Jenkins, Docker, Kubernetes, Shell Scripting, Terraform Professional Attributes: €¢ Ready to work in B Shift (12 PM €“ 10 PM) €¢ A Client facing skills: solid experience working in client facing environments, to be able to build trusted relationships with client stakeholders.€¢ Good critical thinking and problem-solving abilities €¢ Health care knowledge €¢ Good Communication Skills Educational Qualification: Bachelor of Engineering / Bachelor of Technology Preferred candidate profile Please apply if you are an immediate joiner and willing to work for C2H

Role & responsibilities As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. You will oversee the development process and ensure successful project delivery. Roles & Responsibilities: - Expected to perform independently and become an SME.- Required active participation/contribution in team discussions.- Contribute in providing solutions to work-related problems.- Lead the design and development of applications.- Act as the primary point of contact for the project team.- Provide guidance and mentorship to junior team members.- Collaborate with stakeholders to gather requirements and ensure project alignment.- Ensure timely delivery of high-quality software solutions. Professional & Technical Skills: - Must To Have Skills: Proficiency in PySpark, Python (Programming Language), Amazon Web Services (AWS).- Strong understanding of data processing and analysis.- Experience in building scalable and efficient data pipelines.- Knowledge of cloud computing and deployment on AWS infrastructure Preferred candidate profile Please apply if you are an immediate joiner and wiling for C2H

EPAM has presence across 40+ countries globally with 55,000 + professionals & numerous delivery centers, Key locations are North America, Eastern Europe, Central Europe, Western Europe, APAC, Mid East & Development Centers in India (Hyderabad, Pune & Bangalore). Location: Gurgaon/Pune/Hyderabad/Bengaluru/Chennai Work Mode: Hybrid (2-3 days office in a week) Job Description: 5-14 Years of in Big Data & Data related technology experience Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on programming with Python Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming Good understanding of Big Data querying tools, such as Hive, and Impala Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files Good understanding of SQL queries, joins, stored procedures, relational schemas Experience with NoSQL databases, such as HBase, Cassandra, MongoDB Knowledge of ETL techniques and frameworks Performance tuning of Spark Jobs Experience with native Cloud data services AWS/Azure/GCP Ability to lead a team efficiently Experience with designing and implementing Big data solutions Practitioner of AGILE methodology WE OFFER Opportunity to work on technical challenges that may impact across geographies Vast opportunities for self-development: online university, knowledge sharing opportunities globally, learning opportunities through external certifications Opportunity to share your ideas on international platforms Sponsored Tech Talks & Hackathons Possibility to relocate to any EPAM office for short and long-term projects Focused individual development Benefit package: • Health benefits, Medical Benefits• Retirement benefits• Paid time off• Flexible benefits Forums to explore beyond work passion (CSR, photography, painting, sports, etc

loud Data Engineer will be responsible for developing on the data lake platform and all applications on Azure cloud. Good data Engineering, data modeling, Postgres SQL knowledge is a must with Python programming background. The Data Engineer will be responsible for providing design and development solutions for applications in the cloud. • Understand requirements and engage with team to design and deliver projects. Design and implement data lake house projects within azure. Design and develop application lifecycle utilizing Microsoft Azure technologies Participate in design and planning and necessary documentation Participate in Agile ceremonies including daily standups, scrum, retrospectives,demos, code reviews. Hands on with Python and Postgres SQL development and Azure data pipelines Engage with team to develop and deliver cross functional products Key Skills a. Data Engineering and SQL b. Python c. PySpark d. Azure Data lake and ADF e. Databricks f. Postgres database and coding experience g. CICD h. Strong communication • Other Responsibilities: * Document and maintain project artifacts. * Maintain comprehensive knowledge of industry standards, methodologies, processes, and best practices. * Complete training as required for Privacy, Code of Conduct etc. * Promptly report any known or suspected loss, theft or unauthorized disclosure or use of PI to the General Counsel/Chief Compliance Officer or Chief Information Officer. * Adhere to the company's compliance program. * Safeguard the company's intellectual property, information, and assets. * Other duties as assigned. • Minimum Qualifications and Job Requirements: * *Required: Azure data engineering associate and databricks data engineering certification and {color:#ff0000}Postgres SQL experience{color}* * Bachelor's degree in CS. 7 years of hands-on experience in designing and developing distributed data pipelines. 5 years of hands-on experience in Azure data service technologies. 5 years of hands-on experience in Python, SQL, Object oriented programming, ETL and unit testing Experience with data integration with APIs, Web services, Queues Experience with Azure DevOps and CI/CD as well as agile tools and processes including JIRA, confluence. * {color:#FF0000}Working experience with writing SQL queries from scratch{color}

Neudesic, an IBM Company is home to some very smart, talented and motivated people. People who want to work for an innovative company that values their skills and keeps their passions alive with new challenges and opportunities. We have created a culture of innovation that makes Neudesic not only an industry leader, but also a career destination for todays brightest technologists. You can see it in our year-over-year growth, made possible by satisfied employees dedicated to delivering the right solutions to our clients Must Have Skills: Prior experience in ETL, data pipelines, data flow techniques using Azure Data Services Working experience in Python, Scala, PySpark, Azure Data Factory, Azure Data Lake Gen2, Databricks, Azure Synapse and file formats like JSON & Parquet Experience in creating ADF Pipelines to source and process data sets. Experience in creating Databricks notebooks to cleanse, transform and enrich data sets. Good understanding about SQL, Databases, NO-SQL DBs, Data Warehouse, Hadoop and various data storage options on the cloud. Development experience in orchestration of pipelines Experience in deployment and monitoring techniques Working experience with Azure DevOps CI/CD pipelines to deploy Azure resources. Experience in handling operations/Integration with source repository Must have good knowledge on Datawarehouse concepts and Datawarehouse modelling Good to Have Skills: Familiarity with DevOps, Agile Scrum methodologies and CI/CD Domain-driven development exposure Analytical / problem solving skills Strong communication skills Good experience with unit, integration and UAT support Able to design and code reusable components and functions Should able to review design, code & provide review comments with justification Zeal to learn new tool/technologies and adoption Power BI and Data Catalog experience

Role & responsibilities: Write and understand complex SQL queries, including joins, CTE, to extract and manipulate the data. Manage and monitor the data pipelines in Azure Data Factory and Databricks. Manage to understand the integration with Veeva CRM and Salesforce Ensure data quality checks and business level checks in data pipeline of ADF and Databricks. Monitor and trouble shoot data and pipeline related issues, perform root cause analysis and implement corrective actions. Monitoring of system performance and reliability, troubleshooting issues and ensuring data delivery before committed SLA. Strong Understanding of Azure DevOps and Release process (Dev to Ops Handover) Preferred candidate profile : Good communication skills and the ability to work effectively in a team environment. Collaborate with business stakeholders and other technical member to provide operational activities related services. Strong Documentation Skills Note: Work Shift Time will be (11:30 am IST 8:30 pm IST) or (4:30 pm IST to 2:30 am IST)

Experience Required: 8+Years Mode of work: Remote Skills Required: Azure DataBricks, Azure Data Factory, Pyspark, Python, SQL, Spark Notice Period : Immediate Joiners/ Permanent/Contract role (Can join within June) Responsibilities Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and Databricks. Gather and analyse data requirements from business stakeholders and identify opportunities for data-driven insights. Build and optimize data pipelines for data ingestion, processing, and integration using Spark and Databricks. Ensure data quality, integrity, and security throughout all stages of the data lifecycle. Collaborate with cross-functional teams to design and implement data models, schemas, and storage solutions. Optimize data processing and analytics performance by tuning Spark jobs and leveraging Databricks features. Provide technical guidance and expertise to junior data engineers and developers . Stay up to date with emerging trends and technologies in cloud computing, big data, and data engineering. Contribute to the continuous improvement of data engineering processes, tools, and best practices. Requirements: Bachelors or masters degree in computer science, engineering, or a related field. 10+ years of experience as a Data Engineer with a focus on building cloud-based data solutions. Strong experience with cloud platforms such as Azure or AWS. Proficiency in Apache Spark and Databricks for large-scale data processing and analytics. Experience in designing and implementing data processing pipelines using Spark and Databricks. Strong knowledge of SQL and experience with relational and NoSQL databases. Experience with data integration and ETL processes using tools like Apache Airflow or cloud-native orchestration services. Good understanding of data modelling and schema design principles. Experience with data governance and compliance frameworks . Excellent problem-solving and troubleshooting skills. Strong communication and collaboration skills to work effectively in a cross-functional team. Interested candidate can share your resume to OR you can refer your friend to Pavithra.tr@enabledata.com for the quick response.

Role : Azure Data Bricks Engineer Location : PAN India Job Description: Key Responsibilities: 1. Implementation of data pipelines using big data tools, such as Azure Data Factory, PySpark, SparkSQL, SQL , and Azure DevOps, which are critical for building scalable and efficient data workflows. 2. Hands-on PySpark programming . 3. Hands-on with SQL (Stored procedures, Complex SQL queries), data modelling. 4. ETL (Extract, Transform, Load) scripts and workflow configurations. 5. Data validation rules and quality checks. 6. Analysing functional specifications and developing dimensional models. 7. Min. IT experience of 5+ Years Mandatory Skills:- Pyspark, SQL, ADF, Data Warehousing Concept

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : Microsoft Azure Databricks, PySparkMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements. Your typical day will involve creating innovative solutions to address various business needs and collaborating with team members to ensure successful application development. Roles & Responsibilities:- Expected to perform independently and become an SME.- Required active participation/contribution in team discussions.- Contribute in providing solutions to work related problems.- Develop and implement efficient applications to meet business requirements.- Collaborate with team members to ensure successful application development.- Conduct regular code reviews and provide constructive feedback.- Stay updated with the latest technologies and trends in application development.- Assist in troubleshooting and resolving application issues. Professional & Technical Skills: - Must To Have Skills: Proficiency in Databricks Unified Data Analytics Platform.- Good To Have Skills: Experience with Microsoft Azure Databricks, PySpark.- Strong understanding of data processing and analytics.- Experience in building and optimizing data pipelines.- Knowledge of cloud platforms and services for data processing.- Familiarity with data modeling and database design. Additional Information:- The candidate should have a minimum of 3 years of experience in Databricks Unified Data Analytics Platform.- This position is based at our Bengaluru office.- A 15 years full time education is required. Qualification 15 years full time education

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Databricks Unified Data Analytics Platform, Microsoft Azure Databricks, PySpark Good to have skills : NAMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements. Your typical day will involve creating innovative solutions to address various business needs and collaborating with team members to ensure successful project delivery. Roles & Responsibilities:- Expected to perform independently and become an SME.- Required active participation/contribution in team discussions.- Contribute in providing solutions to work-related problems.- Develop and implement software solutions using Databricks Unified Data Analytics Platform.- Collaborate with cross-functional teams to gather and analyze requirements.- Conduct code reviews and provide technical guidance to junior team members.- Troubleshoot and debug applications to ensure optimal performance.- Stay updated on emerging technologies and trends in the field of application development. Professional & Technical Skills: - Must To Have Skills: Proficiency in Databricks Unified Data Analytics Platform, Microsoft Azure Databricks, PySpark.- Strong understanding of cloud computing concepts and experience with Microsoft Azure services.- Experience in developing and deploying scalable applications using cloud-based technologies.- Knowledge of database management systems and data modeling techniques.- Familiarity with Agile methodologies and software development best practices. Additional Information:- The candidate should have a minimum of 3 years of experience in Databricks Unified Data Analytics Platform.- This position is based at our Bengaluru office.- A 15 years full-time education is required. Qualification 15 years full time education

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : Microsoft Azure Databricks, PySparkMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements. Your typical day will involve collaborating with teams to develop innovative solutions and streamline processes. Roles & Responsibilities:- Expected to be an SME- Collaborate and manage the team to perform- Responsible for team decisions- Engage with multiple teams and contribute on key decisions- Provide solutions to problems for their immediate team and across multiple teams- Lead the development and implementation of new applications- Conduct code reviews and ensure coding standards are met- Stay updated on industry trends and best practices Professional & Technical Skills: - Must To Have Skills: Proficiency in Databricks Unified Data Analytics Platform- Good To Have Skills: Experience with PySpark- Strong understanding of data engineering concepts- Experience in building and optimizing data pipelines- Knowledge of cloud platforms like Microsoft Azure- Familiarity with data governance and security practices Additional Information:- The candidate should have a minimum of 5 years of experience in Databricks Unified Data Analytics Platform- This position is based at our Bengaluru office- A 15 years full-time education is required Qualification 15 years full time education

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Microsoft Azure Databricks Good to have skills : Microsoft SQL Server, Microsoft Azure Analytics ServicesMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. You will oversee the development process and ensure successful project delivery. Roles & Responsibilities:- Expected to perform independently and become an SME.- Required active participation/contribution in team discussions.- Contribute in providing solutions to work-related problems.- Lead the design and development of applications.- Act as the primary point of contact for the project team.- Provide guidance and mentorship to junior team members.- Collaborate with stakeholders to gather requirements and ensure project alignment.- Ensure timely delivery of high-quality solutions. Professional & Technical Skills: - Must To Have Skills: Proficiency in Microsoft Azure Data Services, Microsoft SQL Server, Microsoft Azure Analytics Services.- Strong understanding of cloud-based data services.- Experience in designing and implementing data solutions on Azure platform.- Knowledge of data warehousing concepts and ETL processes.- Hands-on experience with Azure data storage and processing services. Additional Information:- The candidate should have a minimum of 2 years of experience in Microsoft Azure Data Services.- This position is based at our Bengaluru office.- A 15 years full-time education is required. Qualification 15 years full time education

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Microsoft Azure Databricks, Java Standard Edition Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements in Pune. You will collaborate with teams to ensure successful project delivery and contribute to key decisions. Roles & Responsibilities:- Expected to be an SME- Collaborate and manage the team to perform- Responsible for team decisions- Engage with multiple teams and contribute on key decisions- Provide solutions to problems for their immediate team and across multiple teams- Lead the development and implementation of new software applications- Conduct code reviews and provide feedback to team members- Stay updated on industry trends and best practices to enhance application development processes Professional & Technical Skills: - Must To Have Skills: Proficiency in Microsoft Azure Databricks, Java Standard Edition- Strong understanding of cloud computing principles- Experience in developing scalable and reliable applications on Azure platform- Knowledge of data processing and analytics tools- Familiarity with DevOps practices and tools Additional Information:- The candidate should have a minimum of 5 years of experience in Microsoft Azure Databricks.- This position is based at our Pune office.- A 15 years full time education is required. Qualification 15 years full time education

Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Microsoft Azure Databricks Good to have skills : NAMinimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. You will oversee the application development process and ensure successful project delivery. Roles & Responsibilities:- Expected to be an SME- Collaborate and manage the team to perform- Responsible for team decisions- Engage with multiple teams and contribute on key decisions- Provide solutions to problems for their immediate team and across multiple teams- Lead the application development process- Ensure successful project delivery- Implement best practices for application design and configuration Professional & Technical Skills: - Must To Have Skills: Proficiency in Microsoft Azure Databricks- Strong understanding of cloud computing concepts- Experience in designing and implementing scalable applications- Knowledge of DevOps practices and tools- Hands-on experience with data integration and ETL processes Additional Information:- The candidate should have a minimum of 7.5 years of experience in Microsoft Azure Databricks- This position is based at our Pune office- A 15 years full-time education is required Qualification 15 years full time education

Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Microsoft Azure Databricks Good to have skills : Microsoft SQL Server, Microsoft Azure Analytics ServicesMinimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your day will involve overseeing the application development process and ensuring successful project delivery. Roles & Responsibilities:- Expected to be an SME- Collaborate and manage the team to perform- Responsible for team decisions- Engage with multiple teams and contribute on key decisions- Provide solutions to problems for their immediate team and across multiple teams- Lead the application development process- Ensure successful project delivery- Mentor and guide team members Professional & Technical Skills: - Must To Have Skills: Proficiency in Microsoft Azure Data Services, Microsoft Azure Analytics Services, Microsoft SQL Server- Good To Have Skills: Experience with Microsoft Azure Analytics Services- Strong understanding of cloud-based data services- Experience in designing and implementing data solutions on Microsoft Azure platform- Knowledge of data warehousing and ETL processes Additional Information:- The candidate should have a minimum of 7.5 years of experience in Microsoft Azure Data Services- This position is based at our Bengaluru office- A 15 years full-time education is required Qualification 15 years full time education

Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Microsoft Azure Databricks Good to have skills : Microsoft SQL Server, Microsoft Azure Analytics ServicesMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your day will involve overseeing the application development process and ensuring successful project delivery. Roles & Responsibilities:- Expected to be an SME- Collaborate and manage the team to perform- Responsible for team decisions- Engage with multiple teams and contribute on key decisions- Provide solutions to problems for their immediate team and across multiple teams- Lead the application development process- Ensure successful project delivery Professional & Technical Skills: - Must To Have Skills: Proficiency in Microsoft Azure Data Services- Good To Have Skills: Experience with Microsoft Azure Analytics Services- Strong understanding of cloud-based data services- Experience with Microsoft SQL Server- Knowledge of data architecture and design principles Additional Information:- The candidate should have a minimum of 5 years of experience in Microsoft Azure Data Services- This position is based at our Bengaluru office- A 15 years full-time education is required Qualification 15 years full time education

Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Microsoft Azure Databricks Good to have skills : Microsoft SQL Server, Microsoft Azure Analytics ServicesMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your day will involve overseeing the application development process and ensuring successful project delivery. Roles & Responsibilities:- Expected to be an SME- Collaborate and manage the team to perform- Responsible for team decisions- Engage with multiple teams and contribute on key decisions- Provide solutions to problems for their immediate team and across multiple teams- Lead the application development process- Ensure successful project delivery Professional & Technical Skills: - Must To Have Skills: Proficiency in Microsoft Azure Data Services- Good To Have Skills: Experience with Microsoft Azure Analytics Services- Strong understanding of cloud-based data services- Experience with Microsoft SQL Server- Knowledge of data architecture and design principles Additional Information:- The candidate should have a minimum of 5 years of experience in Microsoft Azure Data Services- This position is based at our Bengaluru office- A 15 years full-time education is required Qualification 15 years full time education

In this Role, Your Responsibilities Will Be: Develop, test, and maintain high-quality software products by using cutting-edge technologies and best programming practices. Design, develop, and maintain scalable and high-performance data pipelines using Azure Data Factory, Azure Synapse Analytics, and other Azure services. Develop and maintain ETL/ELT processes to ensure the quality, consistency, and accuracy of data. Implement data integration solutions across on-premises, hybrid, and cloud environments. Ensure the security, availability, and performance of enterprise data platforms. Work with relational (SQL Server, Azure SQL) and non-relational databases (Azure Cosmos DB, etc.). Build, test, and deploy data solutions using Azure DevOps, version control, and CI/CD pipelines. Develop and enforce data governance policies and practices to ensure data integrity and security. Perform code reviews and ensure the quality of deliverables from the other developers are as per the standards. Collaborate with multi-functional teams, including designers, developers, and quality assurance engineers to build, refine, and enhance software products. Optimize and troubleshoot large datasets using Python and Azure cloud-native technologies. Create and maintain comprehensive technical documentation.

Databricks, Python, Pyspark, Azure Databricks, SQL, Azure Data Factory, Any cloud (AWS/Azure), any Data Modelling tool (MSBI ,Snowflake ,Google Big query, AWS Redshift ) Shift Timings (Client 1): 6:00 AM IST 2:00 PM IST (OR) Shift Timings (Client 2): 2:00 PM IST 11:00 PM IST Role & responsibilities Preferred candidate profile

Role: Data Engineer Experience: 6-7 Years Location: Bangalore, Pune Notice Period: Immediate Joiners Only Role & responsibilities : Mandate Skills Strong - Pyspark (Programming) Databricks Background and experience: Bachelors Degree in Computer Science or related field 6-7 years of experience in Cloud Data Engineering. Technical and professional skills: We are looking for a flexible, fast learning, technically strong Data Engineer. Expertise is required in the following fields: Proficient in Cloud Services Azure Build ETL and data movement solutions. Migrate data from traditional database systems to Cloud environment Strong hands-on experience for working with Streaming dataset Building Complex Notebook in Databricks to achieve business Transforma tions. Hands-on Experience in Data Refinement using Pyspark and Spark SQL Familiarity with building dataset using Scala. Familiarity with tools such as Jira and GitHub Experience leading agile scrum, sprint planning and review sessions Good communication and interpersonal skills Reach us:If you are interested in this position and meet the above qualifications, please reach out to me directly at swati@cielhr.com and share your updated resume highlighting your relevant experience.

Salary: 8 to 24 LPA Exp: 3 to 7 years Location: Gurgaon (Hybrid) Notice: Immediate to 30 days..!! Job Title: Senior Data Engineer Job Summary: We are looking for an experienced Senior Data Engineer with 5+ years of hands-on experience in cloud data engineering platforms, specifically AWS, Databricks, and Azure. The ideal candidate will play a critical role in designing, building, and maintaining scalable data pipelines and infrastructure to support our analytics and business intelligence initiatives. Key Responsibilities: Design, develop, and optimize scalable data pipelines using AWS services (e.g., S3, Glue, Redshift, Lambda). Build and maintain ETL/ELT workflows leveraging Databricks and Apache Spark for processing large datasets. Work extensively with Azure data services such as Azure Data Lake, Azure Synapse, Azure Data Factory, and Azure Databricks. Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver high-quality data solutions. Ensure data quality, reliability, and security across multiple cloud platforms. Monitor and troubleshoot data pipelines, implement performance tuning, and optimize resource usage. Implement best practices for data governance, metadata management, and documentation. Stay current with emerging cloud data technologies and industry trends to recommend improvements. Required Qualifications: 5+ years of experience in data engineering with strong expertise in AWS , Databricks , and Azure cloud platforms. Hands-on experience with big data processing frameworks, particularly Apache Spark. Proficient in building complex ETL/ELT pipelines and managing data workflows. Strong programming skills in Python, Scala, or Java. Experience working with structured and unstructured data in cloud storage solutions. Knowledge of SQL and experience with relational and NoSQL databases. Familiarity with CI/CD pipelines and DevOps practices in cloud environments. Strong analytical and problem-solving skills with an ability to work independently and in teams. Preferred Skills: Experience with containerization and orchestration tools (Docker, Kubernetes). Familiarity with machine learning pipelines and tools. Knowledge of data modeling, data warehousing, and analytics architecture.

Title: Azure Data Engineer Experience: 3-6 Yrs High level overview Candidates must have a solid understanding of Azure data tech stack. Should be proficient in contributing towards development of ingestion and compute (data transformation) pipelines in large complex Enterprise databases/warehouses using Microsoft Azure Data Technologies. Candidate should have very strong SQL. Design and develop high quality code in accordance with privacy, security, and coding standards guidelines. Job qualifications: Desired Educational Qualification & Technical Skills: [BE/B. Tech/MS/MCA or equivalent] Should have strong programming skills with the ability to write optimized and reusable high-quality code. 3 years experience in building ingestion pipelines using Azure Data Factory, building pipelines for data transformations to compute analytical metrics using SQL/Scala/Python in Azure Big data tech stack (Azure Databricks/Synapse). Solid understanding of data modelling, building facts and dimensions. 3 years of hands-on to SQL in optimized query writing, Performance tuning, Troubleshooting, debugging & development experience. 3 years experience in data visualization in building dashboards/reports using Microsoft Power BI. Dev ops knowledge of code check-ins, building release pipelines for CICD. Desired Soft Skills: Ability to rapidly assimilate new information and new techniques. Display high degree of Confidence and ability to work in ambiguous situations. Strong analytical skills in understanding business requirements

Skills- Azure Data factory, Data Bricks, Scala. Location- PAN India. Experience- 5 years- 12 years

(Looking for immediate joiner who can join within 10- 15 days max). Conceptualize and communicate Data architecture strategy, technical design and technology roadmap for data platform solutions and services. Lead design and development of data architecture to support implementation of large-scale data solutions in Databricks to support multiple use cases (delta lake, cloud Datawarehouse migrations, reporting and analytics). Guiding the organization and development teams on the overall data processes, architectural approaches for data strategy design and implementation. Providing data solutions that enable business intelligence data integrations, data services, self-service analytics, and data-driven digital products and services. Articulating the value proposition of cloud modernization/transformation to stakeholders; creating detailed documentation that empowers other engineers and customers. Architecting the solutions for Databricks-specific use cases (streaming and batch) as well as integrating it into other solution components in the customer architecture landscape. Translating the advanced business data, integration and analytics problems into technical approaches that yield actionable recommendations, across multiple, diverse domains. Implement architectural improvements for existing solutions using legacy modernization and cloud adoptions strategies. Creating Azure Data Factory processes for SQL and NoSQL data pipelines, and work in Azure SQL, to create tables, views, stored procedures, and other database objects. Creating applications through Scala, Python, and Java. Developing Spark tasks for data aggregation and transformation Writing Scala style documents with each code. Identifying technical issues (performance bottlenecks, data discrepancies, system defects) in data and software, perform root cause analysis, and communicate results effectively to the development team and other stakeholders Providing recommendations on opportunities and improvements and participating in technology and analytics proof-of-concepts 8 years of experience with Azure Databricks, Azure Data Factory, and Pyspark 5 years experience with deployment and builds using Azure DevOps 3 years’ experience with Python to extract data from unstructured files, such as XML, PDF, HTML Experience in an agile development environment, or an understanding of the concepts of Agile software development Excellent communication, organization, technical, and project management skills Experience in leading projects relating to cloud modernization, data migration, data warehousing experience with cloud-based data platforms (Databricks/Apache Spark); preferably Databricks certified; experience driving technical workshops with technical and business clients to derive value added services and implementation. Hands-on working knowledge of topics such as data security, messaging patterns, ELT, Data wrangling and cloud computing and proficiency in data integration/EAI and DB technologies, sophisticated analytics tools, programming languages or visualization platforms