Senior Consultant - AI and Data - AWS-Azure Databricks

Role Overview: As a Senior Data Engineer at EY, you will be responsible for ingesting, building, and supporting large-scale data architectures that serve multiple downstream systems and business users. You will collaborate with Data Engineer Leads and Visualization teams to ensure data quality and troubleshoot any issues. Your role will involve cleaning, aggregating, and organizing data from various sources, transferring it to data warehouses, and supporting the development, testing, and maintenance of data pipelines and platforms to enable data quality for business dashboards and tools. Key Responsibilities: - Develop and maintain scalable data pipelines in alignment with ETL principles, utilizing AWS native technologies to support increasing data source volume and complexity. - Define data requirements, gather and mine data, and validate the efficiency of data tools in the Big Data Environment. - Lead the evaluation, implementation, and deployment of emerging tools and processes to enhance productivity. - Implement systems to provide accurate and available data to stakeholders, downstream systems, and business processes. - Collaborate with Business Analysts, Solution Architects, and Data Scientists to design technical architectures for strategic projects and initiatives. - Mentor and coach junior Data Engineers on data standards and practices, fostering a culture of learning and growth. Qualification Required: To qualify for this role, you must have the following essential skillsets: - Bachelor's degree in engineering, Computer Science, Data Science, or a related field. - 6+ years of experience in software development, data engineering, ETL, and analytics reporting development. - Proven track record of designing and implementing complex data solutions. - Proficiency in Python, PySpark, SQL, relational database management systems, Cloud platforms (AWS/Azure), CICD tools, and data architecture principles. - Hands-on experience with Spark Structured Streaming for real-time ETL pipelines. - Strong organizational, communication, problem-solving, and troubleshooting skills. - Ability to work in a fast-paced environment and adapt to changing priorities. Desired Skillsets: - Master's degree in engineering specialized in Computer Science, Data Science, or a related field. - Experience working with pharma clients, particularly in commercial pharma, marketing, and medical affairs. - Knowledge of job orchestration tools, NoSQL databases, and certifications such as Databricks Certified Data Engineer Associate and AWS Certified Data Engineer - Associate. About the Company: EY is dedicated to building a better working world by creating long-term value for clients, people, and society while fostering trust in the capital markets. With diverse teams in over 150 countries, EY leverages data and technology to provide assurance and support clients in growth, transformation, and operations across various sectors.,