1521 Data Lake Jobs - Page 27

Lead Software Engineer - Apache Nifi, Python, PySpark, Hadoop, Cloudera platforms, and Airflow Title and Summary Lead Software Engineer - Apache Nifi, Python, PySpark, Hadoop, Cloudera platforms, and AirflowWho is Mastercard? Mastercard is a global technology company in the payments industry. Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart, and accessible. Using secure data and networks, partnerships and passion, our innovations and solutions help individuals, financial institutions, governments, and businesses realize their greatest potential. Our decency quotient, or DQ, drives our culture and e...

Overview Mastercard is looking for a Senior Software Engineer to join the Account Level Management(ALM) team in our Pune office thats focused on building ALM Services with Data warehouse skills. The Mastercard Account Level Management platform empowers real-time card level decisioning. As consumers progress along their life stages as card holders, with increasing disposable income and more refined preferences, ALM provides services to issuers so they can effectively offer more relevant benefits and rewards at each stage, to drive loyalty and spend. Role o Develop high quality, secure and scalable data pipelines using spark, Scala/Python/Java on Hadoop or object storage like MinIO. o Leverage...

My linkedin linkedin.com/in/yashsharma1608. contract period - 6-12month payroll will be - ASV consulting , my company client - Team Computers Job location - Pune - onsite(WFO) budget - upto 12-13lpa , depending on last (relevant hike) Exprnce - 6+ **************JD is************************** 1)Data Architect or Senior Data Engineer Requirement As discussed, start working on the below requirement. Experience-minimum 6+years or more Immediate joiners for Pune Budget-1Lakh -1.10L per month Required Skills & Qualifications Bachelors or Master’s degree in Computer Science, Information Systems, Data Management, or a related field. Proven experience as a Data Architect or Sr Data Engineer Strong e...

Responsibilities: * Collaborate with cross-functional teams on project delivery. * Design, implement & optimize Azure solutions using certified expertise. * Ensure compliance, security & scalability standards met. Food allowance Health insurance Provident fund Annual bonus

Roles and Responsibilities Design and implement data solutions using Data Architecture principles, including Data Models, Data Warehouses, and Data Lakes. Develop cloud-based data pipelines on AWS/GCP platforms to integrate various data sources into a centralized repository. Ensure effective Data Governance through implementation of policies, procedures, and standards for data management. Collaborate with cross-functional teams to identify business requirements and develop technical roadmaps for data engineering projects. Desired Candidate Profile 7-12 years of experience in Solution Architecting with expertise in Data Architecture, Data Modeling, Data Warehousing, Data Integration, Data Lak...

Job Description Summary This role will play a critical role in supporting the airline customers by designing, developing, and maintaining data pipelines and analytical solutions that drive operational efficiency and fuel savings. This role is responsible for ensuring high-quality data ingestion, transformation, and validation from diverse operational sources, enabling the team to monitor, analyze, and optimize fuel consumption across the airlines fleet. The engineer will work closely with fuel analysts, operations teams, and IT to deliver accurate, timely insights that support decision-making, sustainability goals, and cost reduction initiatives. Job Description Site Overview Role Overview B...

About your team The Data Platform team manage the products and technical infrastructure that underpin the use of data at Fidelity - Databases (Oracle, SQL Server, PostgreSQL, Snowflake), data streaming (Kafka, Snaplogic), data security, data analytics (PowerBI, Oracle Analytics Cloud), data management, and more. We provide both cloud-based and on-premises solutions as well as automation and self-service tools. The company predominately uses AWS, but we also deploy on Azure and Oracle Cloud in certain situations. About your role You role will be to use your skills (and the many skills you will acquire on the job) to develop our cloud solutions for the products and services we look after. We o...

As a Senior Lead Software Engineer at JPMorgan Chase within the Consumer and Community Banking, you will be a crucial member of an agile team dedicated to enhancing, building, and delivering cutting-edge technology products in a secure, stable, and scalable manner. Your role involves driving significant business impact through your expertise and contributions, applying deep technical knowledge to solve a diverse range of challenges across multiple technologies and applications. You will regularly provide technical guidance and direction to support the business, technical teams, contractors, and vendors. Your responsibilities include developing secure, high-quality production code, reviewing ...

As an Azure Data Engineer specializing in Microsoft Fabric (Data Lake) based in Mumbai, you should have a minimum of 4 years of experience in the field, with at least 2 years dedicated to working with Microsoft Fabric technologies. Your expertise in Azure services is key, specifically in Data Lake, Synapse Analytics, Data Factory, Azure Storage, and Azure SQL. Your responsibilities will involve data modeling, ETL/ELT processes, and data integration patterns. It is essential to have experience in Power BI integration for effective data visualization. Proficiency in SQL, Python, or PySpark for data transformations is required for this role. A solid understanding of data governance, security, a...

As a Data Engineer, you will be responsible for designing and implementing next-generation data pipelines, analytics, and Business Intelligence solutions. You will manage various AWS resources such as RDS, EMR, MWAA, and Lambda. Your primary focus will be on developing high-quality data architecture and pipelines to support business analysts and data scientists. Collaborating with other technology teams, you will extract, transform, and load data from diverse sources. Additionally, you will work towards enhancing reporting and analysis processes by automating self-service support for customers. You should have a solid background with 7-10 years of data engineering experience, including exper...

Primary Responsibilities: Should be Curious and Quick learner for Tech needed for the project Should be able to Explore and Suggest new Ideas/Tech for the project Should be able to understand the feature requirements from the product and be able to come up with Technical Solutions keeping in mind the Scalability, Extensibility, Security, Cost etc. Come up with a plan break down the feature in workable stories Work closely with Product & Development team The Delivered feature should be of Quality with low defect rate Work collaboratively with the team and should be able to help others on technical issues or functional questions they might have Should be able to own a feature and run through w...

The Enterprise Architect will work with the Enterprise Architecture Automation team and directly collaborate with multiple internal teams such as Sales, Finance, Marketing, and Customer Success; this role also interacts with other engineering functions such as Enterprise Systems & IT Operations, Technology Operations, Cybersecurity, and any other parties involved throughout the architecting/design and rollout of those solutions. You will be part of the PitchBook Architecture function reporting to the PitchBook Director of Software Architecture, and working closely with our Senior Director of Enterprise Technology. As an Enterprise Architect within the PitchBook Engineering organization, you ...

Manadatory skills : Strong in MS SQL and SSIS , Data Lake, Azure SQL, SSIS, ADF , Lead experience , Good communication Notice period : immediate - 20 days

Project Role : Software Development Lead Project Role Description : Develop and configure software systems either end-to-end or for a specific stage of product lifecycle. Apply knowledge of technologies, applications, methodologies, processes and tools to support a client, project or entity. Must have skills : Manufacturing Engineering MES Good to have skills : NAMinimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Software Development Lead, you will be responsible for developing and configuring software systems, either end-to-end or for specific stages of the product lifecycle. Your typical day will involve collaborating with v...

We are hiring for a prestigious client for below location. Location: Bangalore , Pune , Chennai , Kolkata , Gurugram (Hybrid work mode) Skills required: Project Management, Snowflake/ GCP, Data engineer Background Only immediate or Serving candidates preferred. more than 30 days notice Kindly don't APPLY. Only B.Tech Candidates. with Excellent communication skills. Budget Max : 36 LPA and depends on previous compensation. Preferred candidate profile Experience as a software developer and as a team lead in the data engineering space. Engineering manager experience would be an added advantage. Atleast 2 years of experience developing data solutions using any data engineering methods. Has worki...

About The Role Key Responsibilities: Provide expert-level support for Azure Machine Learning, OpenAI, Cognitive Services, Synapse, and Data Factory . Troubleshoot real-time data integration pipelines, model deployments, inference performance, and security configurations. Guide customers in implementing Responsible AI , security governance, and model versioning best practices. Collaborate with engineering teams to resolve service issues and release platform updates. Proactively identify patterns in customer challenges and build reusable diagnostics or automations. Work with cross-functional teams (Product, Engineering, Customer Success) to improve support processes and tooling. Skillset: Stro...

Dear , Greetings from Tekskills India Pvt Ltd (CMMI Level3 Company) !!! Pleasure mailing you ! We have an immediate job requirement. Below are the job details, Please go through below job description.If you are suitable and interested, please send your updated word resume inline to the JD & share the below submission details. Please Share the Below Details: Passport Size Photo : Offer letter/ Relieving letter screenshot/ LWD email screenshot: Total Experience: Relevant Experience: Current Location: Preferred Location: Current CTC: Expected CTC: Notice Period: DOB: Offers/ Pipelines: Job title: - Data Engineer Location: Bengaluru & Pune (Relocation is considered) Shift Timings: 4:00 PM to 1:0...

Must have skills : 15 years of experience in design and delivery of Distributed Systems capable of handling petabytes of data in a distributed environment. 10 years of experience in the development of Data Lakes with Data Ingestion from disparate data sources, including relational databases, flat files, APIs, and streaming data. Experience in providing Design and development of Data Platforms and data ingestion from disparate data sources into the cloud. Expertise in core AWS Services including AWS IAM, VPC, EC2, EKS/ECS, S3, RDS, DMS, Lambda, CloudWatch, CloudFormation, CloudTrail, CloudWatch. Proficiency in programming languages like Python and PySpark to ensure efficient data processing. ...

Must have skills : 15 years of experience in design and delivery of Distributed Systems capable of handling petabytes of data in a distributed environment. 10 years of experience in the development of Data Lakes with Data Ingestion from disparate data sources, including relational databases, flat files, APIs, and streaming data. Experience in providing Design and development of Data Platforms and data ingestion from disparate data sources into the cloud. Expertise in core AWS Services including AWS IAM, VPC, EC2, EKS/ECS, S3, RDS, DMS, Lambda, CloudWatch, CloudFormation, CloudTrail, CloudWatch. Proficiency in programming languages like Python and PySpark to ensure efficient data processing. ...

Role: Software Engineer, Direct Platform Quality, EDP The Group : Morningstar’s The Enterprise Data Platform (EDP) is the central nervous system of consolidating, normalizing, calculating, and distributing Morningstar’s high-quality data and capabilities around that data for internal and external consumers. EDP is the data provider as well as business enabler. The individuals in this group thrive to enable faster business growth, better scale and improve customer experience by maximizing full commercial and brand value of data sets in their entirety. The Role : You will be accountable for working with a quality engineering team dedicated to implementing and maintaining data quality solutions...

About KPMG in India KPMG entities in India are professional services firm(s) affiliated with KPMG International Limited, established in India in August 1993. Our professionals leverage the global network of firms and possess in-depth knowledge of local laws, regulations, markets, and competition. With offices spread across India in cities like Ahmedabad, Bengaluru, Chandigarh, Chennai, Gurugram, Jaipur, Hyderabad, Kochi, Kolkata, Mumbai, Noida, Pune, Vadodara, and Vijayawada, KPMG entities in India offer services to national and international clients across various sectors. We aim to provide rapid, performance-based, industry-focused, and technology-enabled services that demonstrate our unde...

The ideal candidate for this role will have a Bachelor's degree in Information Technology, Computer Science Engineering, or related fields. You should have at least 5 years of experience in infrastructure management and systems administration. Your primary responsibility will be to provide technical support for Azure cloud services. We are looking for a Cloud Engineer with a strong background in both applications and infrastructure. You should have hands-on experience with Azure services such as Data Lake, Data Factory, and Databricks. Proficiency in setting up Data Warehouse, ETL processes, Reporting, and Analytics will be advantageous. Experience with tools like Control-M, SAS, SunGL, and ...

Your opportunity to make a real impact and shape the future of financial services is waiting for you. Let's push the boundaries of what's possible together. As a Senior Director of Software Engineering at JPMorgan Chase within the Consumer and Community Banking division, you will be responsible for leading various technical domains, overseeing the activities of multiple departments, and fostering cross-functional collaboration. Your technical expertise will be utilized across different teams to promote the adoption and implementation of advanced technical methods, helping the firm stay ahead of industry trends, best practices, and technological advancements. Leads multiple technology and pro...

If you are a smart & passionate team player - then this Senior Data Engineer [Snowflake] opportunity is for you! We at IMRIEL(An Allata Company) are looking for a Senior Data Engineer to implement methods to improve data reliability and quality. You will combine raw information from different sources to create consistent and machine-readable formats. You will also develop and test architectures that enable data extraction and transformation for predictive or prescriptive modeling. Resourcefulness is a necessary skill in this role. If you truly love gaining new technical knowledge and can add more awesomeness to the team, you are eligible! Experience : 5 to 8 years. Location : Vadodara & Pune...

Support Development, and maintain automated test frameworks, tools, and test cases for Data Engineering and Data Warehouse applications. Collaborate with cross-functional teams, including software developers, data engineers, and data analysts, to ensure comprehensive testing coverage and adherence to quality standards. Conduct thorough testing of data pipelines, ETL processes, and data transformations using Big Data technologies. Apply your knowledge of Data Warehouse/Data Lake methodologies and best practices to validate the accuracy, completeness, and performance of our data storage and retrieval systems. Identify, document, and track software defects, working closely with the development ...