1693 Data Engineering Jobs - Page 50

Our client is a global leader in the aviation sector, driving a digital-first transformation powered by cloud technologies, data innovation, and machine learning. With a bold vision to redefine how data empowers smarter decisions, they are building a modern engineering ecosystem that fuels business agility and growth at scale. At the heart of this journey is a vibrant, inclusive, and forward-thinking technology team that thrives on curiosity, collaboration, and continuous learning. From cloud-native architectures to real-time data pipelines, team members are shaping solutions that impact millions of livesall while working in an environment that values diversity, wellbeing, and career development. The Job We are looking for a Senior Manager, Data Engineering & Development to join United Data Engineering team. As a senior manager, you will play a key role leading a team of engineers who will look to you for advice on technical and business challenges. You will guide the team and insist on highest standards for quality, maintainability, and performance. You will ensure that engineering best practices are followed and the data systems are designed to be highly available, scalable. You will set the strategic direction as well as own and prioritize goals for your team. Successful candidates will be high-bandwidth leaders who can cut through the noise, simplify relentlessly, deliver results, and build great teams around themselves. Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies Work with a team of developers with deep experience in big data technology, distributed systems and business intelligence tools Utilize programming languages like Java, Scala, Python and NoSQL databases and Cloud based data warehousing services such as Redshift Utilize tools like Power BI Desktop, DAX Studio, and Power Query to develop and optimize data models in PowerBI. Lead reviews of PBI models, solution strategies to ensure best practices to integrate BI solutions with current data platforms. Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community Perform unit tests and conducting code reviews with other team members to make sure application development and architecture standards are met, code is developed using best practices Your Profile Ideal candidates will have expertise in all phases of data & software lifecycle management processes. BS Computer Science, Engineering or related discipline or equivalent experience required • Cloudera/Hadoop Certification or AWS Developer/Big Data Analytics certification a plus 10+ years of experience in managing technical teams and projects 5+ years of experience leading an Engineering team familiar with large cloud environments, Big Data technologies 5+ years of development experience using Java, Python, Scala 4+ years experience with Batch and real time streaming Subject matter expert in Big Data, AWS technology Knowledge of PowerBI star schema and data modelling techniques Good understanding of all aspects of data architectures, data dictionaries, and data lineages in a large-scale enterprise environment Demonstrable experience with distributed data warehousing and data processing systems such as Redshift Deep knowledge of traditional data warehousing (models, practices, and processes), cloud based data warehouses, data lakes, and data streaming Exceptional system design abilities, including high-scale ETL/ELT pipelines including leveraging python Experience with or solid understanding of data pipelines and event-driven architectures (Kinesis, Kafka) Proficient with AWS Cloud technologies (i.e. S3, Lambda, Dynamo, EC2), Python, Spark, and Hadoop

Roles and Responsibilities: You are detailed reviewing and analyzing structured, semi-structured and unstructured data sources for quality, completeness, and business value. You design, architect, implement and test rapid prototypes that demonstrate value of the data and present them to diverse audiences. You participate in early state design and feature definition activities. Responsible for implementing robust data pipeline using Microsoft, Databricks Stack Responsible for creating reusable and scalable data pipelines. You are a Team-Player, collaborating with team members across multiple engineering teams to support the integration of proven prototypes into core intelligence products. You have strong communication skills to effectively convey complex data insights to non-technical stakeholders. Critical Skills to Possess: Skills: Advanced working knowledge and experience with relational and non-relational databases. Advanced working knowledge and experience with API data providers Experience building and optimizing Big Data pipelines, architectures, and datasets. Strong analytic skills related to working with structured and unstructured datasets. Hands-on experience in Azure Databricks utilizing Spark to develop ETL pipelines. Strong proficiency in data analysis, manipulation, and statistical modeling using tools like Spark, Python, Scala, SQL, or similar languages. Strong experience in Azure Data Lake Storage Gen2, Azure Data Factory, Databricks, Event Hub, Azure Synapse. Familiarity with several of the following technologies: Event Hub, Docker, Azure Kubernetes Service, Azure DWH, API Azure, Azure Function, Power BI, Azure Cognitive Services. Azure DevOps experience to deploy the data pipelines through CI/CD. Roles and Responsibilities Skills: Azure Databricks, Azure Datafactory, Big Data Pipelines, Pyspark, Azure Synapse, Azure DevOps, Azure Data Lake Storage Gen2, Event Hub, Azure DWH, API Azure. Experience: Minimum 5-7 years of practical experience as Data Engineer. Azure cloud stack in-production experience. Preferred Qualifications: BS degree in Computer Science or Engineering or equivalent experience

We are hiring "Project Manager- Data Engineering" for one of my Product company @ Pune location EXP-5+ Mode-Permanent NP-can join June Mandatory skills- Python, SQL, GenAI, Data Engineering, Project management, Team leading experience

We are looking to hire a Senior Manager, Customer Analytics to join our team in Bangalore! Reporting to the Director of Customer Insights & Analytics, youll lead a high-performing team of analysts and Subject matter authorities to deliver customer-centric insights, business intelligence, and data-driven strategies. In this role, youll act as a strategic partner to global collaborators across Marketing, Sales, and CRM, helping drive improvements in customer experience, marketing performance, and commercial outcomes. Youll also play a key role in mentoring talent, crafting analytics processes, and scaling insight capabilities spanning multiple business units. What Youll Do: Lead and mentor a team of 68 analysts and data professionals, encouraging a collaborative, high-performance culture Drive customer analytics initiatives including segmentation, cohort analysis, and performance tracking Partner with global Sales, Marketing, CRM, and Data leaders to align on priorities and deliver actionable insights Translate sophisticated data into clear recommendations and compelling narratives for non-technical collaborators Influence senior-level decision-making through data-driven storytelling and strategic insight Coordinate end-to-end delivery of analytics projects across divisions, ensuring quality and business relevance Improve and standardize analytics tools, processes, and documentation for scalability and consistency Collaborate with Tech and Data Engineering teams to improve customer data infrastructure and analytics capabilities Serve as a trusted advisor to senior leadership on demonstrating customer intelligence for business growth Leverage AI to improve team productivity and analytics efficiency What Youll Bring: Extensive experience in data analytics, customer insights, or marketing analytics, with a proven record in team leadership and management Optimally led customer-centric analytics initiatives within global or matrixed organizations Sophisticated analytical skills with hands-on expertise in SQL and tools like Python, R, or similar statistical platforms Strong proficiency in BI tools such as Power BI and Google Data Studio Familiar with cloud-based data platforms like Google BigQuery and similar analytics stacks Skilled in communicating sophisticated analytical insights clearly and persuasively to senior collaborators Business-savvy approach with the ability to translate data into impactful strategic actions Proven success in mentoring and developing high-performing analytics teams

Job Title : Data Engineer Work Location : India, Pune / Hyderabad (Hybrid) Responsibilities include: Design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data. Develop data pipelines to extract and transform data in near real time using cloud native technologies. Implement data validation and quality checks to ensure accuracy and consistency. Monitor system performance, troubleshoot issues, and implement optimizations to enhance reliability and ePiciency. Collaborate with business users, analysts, and other stakeholders to understand data requirements and deliver tailored solutions. Document technical designs, workflows, and best practices to facilitate knowledge sharing and maintain system documentation. Provide technical guidance and support to team members and stakeholders as needed. Desirable Competencies: 8+ years of work experience Proficiency in writing complex SQL queries on MPP systems (Snowflake/Redshift) Experience in Databricks and Delta tables. Data Engineering experience with Spark/Scala/Python Experience in Microsoft Azure stack (Azure Storage Accounts, Data Factory and Databricks). Experience in Azure DevOps and CI/CD pipelines. Working knowledge of Python Feel comfortable participating in 2-week sprint development cycles. About Us Founded in 1956, Williams-Sonoma Inc. is the premier specialty retailer of high-quality products for the kitchen and home in the United States. Today, Williams-Sonoma, Inc. is one of the United States' largest e-commerce retailers with some of the best known and most beloved brands in home furnishings. Our family of brands are Williams-Sonoma, Pottery Barn, Pottery Barn Kids, Pottery Barn Teens, West Elm, Williams-Sonoma Home, Rejuvenation, GreenRow and Mark and Graham. We currently operate retail stores globally. Our products are also available to customers through our catalogues and online worldwide. Williams-Sonoma has established a technology center in Pune, India to enhance its global operations. The India Technology Center serves as a critical hub for innovation and focuses on developing cutting-edge solutions in areas such as e-commerce, supply chain optimization, and customer experience management. By integrating advanced technologies like artificial intelligence, data analytics, and machine learning, the India Technology Center plays a crucial role in accelerating Williams-Sonoma's growth and maintaining its competitive edge in the global market.

ABOUT THE ROLE The Global Quality Analytics and Innovation team leads the digital transformation and innovation effort throughout Amgen’s Quality organization. We are at the forefront of developing and rolling out data-centric digital tools, employing automation, artificial intelligence (AI), and generative AI to drive end-to-end quality transformation. We are seeking a highly motivated and experienced Senior Data Scientist with a strong background in Generative AI, Large Language Models (LLMs), and MLOps, along with an understanding for Quality in regulated environments (e.g., GxP). This role will play a key part in designing, developing, and deploying scalable AI/ML solutions to drive innovation, efficiency, and regulatory compliance across the organization. You will collaborate with cross-functional teams, including software engineers, data engineers, business stakeholders, and quality professionals to deliver AI-driven capabilities that support strategic business objectives. The ideal candidate is an analytical thinker with excellent technical depth, communication skills, and the ability to thrive in a fast-paced, agile environment. Key Responsibilities Design, build, and deploy generative AI and LLM-based applications using frameworks such as LangChain, LlamaIndex, and others. Engineer reusable and effective prompts for LLMs like OpenAI GPT-4, Anthropic Claude, etc. Develop and maintain evaluation metrics and frameworks for prompt engineering. Collaborate with business stakeholders to identify AI/ML opportunities, ensuring alignment between technical solutions and business goals. Lead the development of MLOps pipelines for model deployment, monitoring, and lifecycle management. Conduct data quality assessments, data cleansing, and ingestion of unstructured documents into vector databases. Build retrieval algorithms for relevant data identification to support LLMs and AI applications. Ensure AI/ML development complies with GxP and other regulatory standards, fostering a strong Quality culture. Partner with global and local teams to support regulatory inspection readiness and future technological capabilities in AI. Share insights and findings with team members in an Agile (SAFe) environment. Basic Qualifications Doctorate degree OR Master’s degree and 4–6 years of experience in Software Engineering, Data Science, or ML Engineering OR Bachelor’s degree and 6–8 years of experience OR Diploma and 10–12 years of experience Preferred Qualifications Proven experience developing and deploying LLM applications. Strong foundation in ML algorithms, data science workflows, and NLP. Expertise in Python and ML libraries (e.g., TensorFlow, PyTorch, Scikit-learn). Familiarity with MLOps tools (e.g., MLflow, CI/CD, version control). Experience with cloud platforms (AWS, Azure, GCP) and tools like Spark, Databricks. Understanding of RESTful APIs and frameworks like FastAPI. Experience with BI and visualization tools (e.g., Tableau, Streamlit, Dash). Knowledge of GxP compliance and experience working in regulated environments. Domain experience in healthcare, biotech, or life sciences is a plus. Strong communication skills with the ability to explain complex topics to diverse audiences. High degree of initiative, self-motivation, and ability to work in global teams.

JOB SUMMARY This Data Scientist creates and implements advanced analytics models and solutions to yield predictive and prescriptive insights from large volumes of structured and unstructured data. This position works with a team responsible for the research and implementation of predictive requirements by leveraging industry standard machine learning and data visualization tools to draw insights that empower confident decisions and product creation. This position leverages emerging tools and technologies available in On-prem and Cloud environments. This position utilizes industry standard machine learning and data visualization tools to transform data and analytics requirements into predictive solutions and provide data literacy on a range of machine learning systems at UPS. This position identifies opportunities for driving descriptive to predictive and prescriptive solutions, which become inputs to department and project teams on their decisions supporting projects. RESPONSIBILITIES Defines key data sources from UPS and external sources to deliver models. Develops and implements pipelines that facilitates data cleansing, data transformations, data enrichments from multiple sources (internal and external) that serve as inputs for data and analytics systems. For larger teams, works with data engineering teams to validate and test data and model pipelines identified during proof of concepts. Develops data design based on the exploratory analysis of large amounts of data to discover trends and patterns that meet stated business needs. Defines model key performance indicator (KPI) expectations and validation, testing, and re-training of existing models to meet business objectives. Reviews and creates repeatable solutions through written project documentation, process flowcharts, logs, and commented clean code to produce datasets that can be used in analytics and/or predictive modeling. Synthesizes insights and documents findings through clear and concise presentations and reports to stakeholders. Presents operationalized analytic findings and provides recommendations. Incorporates best practices on the use of statistical modeling, machine learning algorithms, distributed computing, cloud-based AI technologies, and run time performance tuning with the goal of deployment and market introduction. Leverages emerging tools and technologies together with the use of open-source or vendor products in the creation and delivery of insights that support predictive and prescriptive solutions. QUALIFICATIONS Expertise in R, SQL, Python and/or any other high-level languages. Exploratory data analysis (EDA), data engineering and development of advanced analytics models. Experience in development of AI and ML using platforms like VertexAI, Databricks or Sagemaker, and familiarity with available frameworks like PyTorch, Tensorflow and Keras. Experience applying models from small to medium scaled problems. Strong analytical skills and attention to detail. Able to engage key business and executive-level stakeholders to translate business problems to a high-level analytics solution approach. Expertise with statistical techniques, machine learning, and/or operations research and their application in business. Deep understanding of data management pipelines and experience in launching moderate scale advanced analytics projects in production. Demonstrated experience in Cloud-AI technologies and knowledge of environments both in Linux/Unix and Windows. Experience implementing open-source technologies and cloud services; with or without the use of enterprise data science platforms. Core AI Machine Learning knowledge and application in supervised and unsupervised learning domains. Familiarity with Java or C++ is a plus. Solid oral and written communication skills, especially around analytical concepts and methods. Ability to communicate data through a story framework to convey data-driven results to technical and non-technical audiences. Masters Degree in a quantitative field of mathematics, computer science, physics, economics, engineering, statistics (operations research, quantitative social science, etc.), international equivalent, or equivalent job experience.

locationsIndia, Bangalore time typeFull time posted onPosted 12 Days Ago job requisition idJR0273871 Job Details: About The Role : About the Role: Join our innovative and inclusive Logic Technology Development team as a TD AI and Analytics Engineer, where diverse talents come together to push the boundaries of semiconductor technology. You will have the opportunity to work in one of the world's most advanced cleanroom facilities, designing, executing, and analyzing experiments to meet engineering specifications for our cutting-edge processes. This role offers a unique chance to learn and operate a manufacturing line, integrating the many individual steps necessary for the production of complex microprocessors. What We Offer: We are dedicated to creating a collaborative, supportive, and exciting environment where diverse perspectives drive exceptional results. At Intel, you will have the opportunity to transform technology and contribute to a better future by delivering innovative products. Learn more about Intel Corporation's Core Values here. Benefits: We offer a comprehensive benefits package designed to support a healthy and fulfilling life. This includes excellent medical plans, wellness programs, recreational activities, generous time off, discounts on various products and services, and many more creative rewards that make Intel a great place to work. Discover more about our amazing benefits here. About the Logic Technology Development (LTD) TD Intel Foundry AI and Analytics Innovation Organization: Intel Foundry TD's AI and Analytics Innovation office is committed to providing a competitive advantage through End-to-End AI and Analytics Solutions, driving Intel's ambitious IDM2.0 goals. Our team is seeking an engineer with a background in Data Engineering, Software Engineering, or Data Science to support and develop modern AI/ML solutions. Explore what life is like inside Intel here. Key Responsibilities: As an Engineer in the TD AI office, you will collaborate with Intel's factory automation organization and Foundry TD's functional areas to support and develop modern AI/ML solutions. Your primary responsibilities will include. Developing software and data engineering solutions for in-house AI/ML products. Enhancing existing ML platforms and devising MLOps capabilities. Understanding existing data structures in factory automation systems and building data pipelines connecting different systems. Testing and supporting full-stack big data engineering systems. Developing data ingestion pipelines, data access APIs, and services, monitoring and maintaining deployment environments and platforms, creating technical documentation, and collaborating with peers/engineering teams to streamline solution development, validation, and deployment. Managing factory big data interaction with cloud environments, ORACLE, SQL, Python, Software architecture, and MLOps. Interfacing with process and integration functional area analytics teams and customers using advanced automated process control systems. Qualifications: Minimum Qualifications: Master's or PhD degree in Computer Science, Computer Engineering, or a related Science/Engineering discipline. 3+ years of experience in data engineering/software development and knowledge in Spark, NiFi, Hadoop, HBase, S3 object storage, Kubernetes, REST APIs, and services. Intermediate to advanced English proficiency (both verbal and written). Preferred Qualifications: 2+ years in data analytics and machine learning (Python, R, JMP, etc.) and relational databases (SQL). 2+ years in a technical leadership role. 3+ months of working knowledge with CI/CD (Continuous Integration/Continuous Deployment) and proficiency with GitHub and GitHub Actions. Prior interaction with factory automation systems. Application Process :By applying to this posting, your resume and profile will become visible to Intel recruiters, allowing them to consider you for current and future job openings aligned with the skills and positions mentioned above. We are constantly working towards a more connected and intelligent future, and we need your help. Change tomorrow. Start today. Job Type: Experienced Hire Shift: Shift 1 (India) Primary Location: India, Bangalore Additional Locations: Business group: As the world's largest chip manufacturer, Intel strives to make every facet of semiconductor manufacturing state-of-the-art -- from semiconductor process development and manufacturing, through yield improvement to packaging, final test and optimization, and world class Supply Chain and facilities support. Employees in theTechnology Development and Manufacturing Groupare part of a worldwide network of design, development, manufacturing, and assembly/test facilities, all focused on utilizing the power of Moores Law to bring smart, connected devices to every person on Earth. Posting Statement: All qualified applicants will receive consideration for employment without regard to race, color, religion, religious creed, sex, national origin, ancestry, age, physical or mental disability, medical condition, genetic information, military and veteran status, marital status, pregnancy, gender, gender expression, gender identity, sexual orientation, or any other characteristic protected by local law, regulation, or ordinance. Position of Trust N/A Work Model for this Role This role will be eligible for our hybrid work model which allows employees to split their time between working on-site at their assigned Intel site and off-site. *

CACI International Inc is an American multinational professional services and information technology company headquartered in Northern Virginia. CACI provides expertise and technology to enterprise and mission customers in support of national security missions and government transformation for defense, intelligence, and civilian customers. CACI has approximately 23,000 employees worldwide. Headquartered in London, CACI Ltd is a wholly owned subsidiary of CACI International Inc., a publicly listed company on the NYSE with annual revenue in excess of US $6.2bn. Founded in 2022, CACI India is an exciting, growing and progressive business unit of CACI Ltd. CACI Ltd currently has over 2000 intelligent professionals and are now adding many more from our Hyderabad and Pune offices. Through a rigorous emphasis on quality, the CACI India has grown considerably to become one of the UKs most well-respected Technology centres. About Data Platform: The Data Platform will be built and managed as a Product” to support a Data Mesh organization. The Data Platform focusses on enabling decentralized management, processing, analysis and delivery of data, while enforcing corporate wide federated governance on data, and project environments across business domains. The goal is to empower multiple teams to create and manage high integrity data and data products that are analytics and AI ready, and consumed internally and externally. What does a Data Infrastructure Engineer do? A Data Infrastructure Engineer will be responsible to develop, maintain and monitor the data platform infrastructure and operations. The infrastructure and pipelines you build will support data processing, data analytics, data science and data management across the CACI business. The data platform infrastructure will conform to a zero trust, least privilege architecture, with a strict adherence to data and infrastructure governance and control in a multi-account, multi-region AWS environment. You will use Infrastructure as Code and CI/CD to continuously improve, evolve and repair the platform. You will be able to design architectures and create re-useable solutions to reflect the business needs. Responsibilities will include: Collaborating across CACI departments to develop and maintain the data platform Building infrastructure and data architectures in Cloud Formation, and SAM. Designing and implementing data processing environments and integrations using AWS PaaS such as Glue, EMR, Sagemaker, Redshift, Aurora and Snowflake Building data processing and analytics pipelines as code, using python, SQL, PySpark, spark, CloudFormation, lambda, step functions, Apache Airflow Monitoring and reporting on the data platform performance, usage and security Designing and applying security and access control architectures to secure sensitive data You will have: 8+ years of experience in a Data Engineering role. Strong experience and knowledge of data architectures implemented in AWS using native AWS services such as S3, DataZone, Glue, EMR, Sagemaker, Aurora and Redshift. Experience administrating databases and data platforms Good coding discipline in terms of style, structure, versioning, documentation and unit tests Strong proficiency in Cloud Formation, Python and SQL Knowledge and experience of relational databases such as Postgres, Redshift Experience using Git for code versioning, and lifecycle management Experience operating to Agile principles and ceremonies Hands-on experience with CI/CD tools such as GitLab Strong problem-solving skills and ability to work independently or in a team environment. Excellent communication and collaboration skills. A keen eye for detail, and a passion for accuracy and correctness in numbers Whilst not essential, the following skills would also be useful: Experience using Jira, or other agile project management and issue tracking software Experience with Snowflake Experience with Spatial Data Processing

About the Role - We are seeking a highly skilled and experienced Senior Azure Databricks Engineer to join our dynamic data engineering team. - As a Senior Azure Databricks Engineer, you will play a critical role in designing, developing, and implementing data solutions on the Azure Databricks platform. - You will be responsible for building and maintaining high-performance data pipelines, transforming raw data into valuable insights, and ensuring data quality and reliability. Key Responsibilities - Design, develop, and implement data pipelines and ETL/ELT processes using Azure Databricks. - Develop and optimize Spark applications using Scala or Python for data ingestion, transformation, and analysis. - Leverage Delta Lake for data versioning, ACID transactions, and data sharing. - Utilize Delta Live Tables for building robust and reliable data pipelines. - Design and implement data models for data warehousing and data lakes. - Optimize data structures and schemas for performance and query efficiency. - Ensure data quality and integrity throughout the data lifecycle. - Integrate Azure Databricks with other Azure services (e.g., Azure Data Factory, Azure Synapse Analytics, Azure Blob Storage). - Leverage cloud-based data services to enhance data processing and analysis capabilities. Performance Optimization & Troubleshooting - Monitor and analyze data pipeline performance. - Identify and troubleshoot performance bottlenecks. - Optimize data processing jobs for speed and efficiency. - Collaborate effectively with data engineers, data scientists, data analysts, and other stakeholders. - Communicate technical information clearly and concisely. - Participate in code reviews and contribute to the improvement of development processes. Qualifications Essential - 5+ years of experience in data engineering, with at least 2 years of hands-on experience with Azure Databricks. - Strong proficiency in Python and SQL. - Expertise in Apache Spark and its core concepts (RDDs, DataFrames, Datasets). - In-depth knowledge of Delta Lake and its features (e.g., ACID transactions, time travel). - Experience with data warehousing concepts and ETL/ELT processes. - Strong analytical and problem-solving skills. - Excellent communication and interpersonal skills. - Bachelor's degree in Computer Science, Computer Engineering, or a related field.

: As a Python Developer, you will play a critical role in our software development and data engineering initiatives. You will work closely with data engineers, architects, and other developers to build and maintain our applications and data pipelines. Your expertise in Python development, API design, and cloud technologies will be essential to your success. Responsibilities : - Design, develop, and maintain applications using the latest Python frameworks and technologies (Django, Flask, FastAPI). - Utilize Python libraries and tools (Pandas, NumPy, SQLAlchemy) for data manipulation and analysis. - Develop and maintain RESTful APIs, ensuring security, authentication, and authorization (OAuth, JWT). - Deploy, manage, and scale applications on AWS services (EC2, S3, RDS, Lambda). - Utilize infrastructure-as-code tools (Terraform, CloudFormation) for infrastructure management (Good to have). - Design and develop database solutions using PL/SQL (Packages, Functions, Ref cursors). - Implement data normalization and Oracle performance optimization techniques. - Design and develop data warehouse solutions, including data marts and ODS concepts. - Implement low-level design of warehouse solutions. - Work with Kubernetes for container orchestration, deploying, managing, and scaling applications on Kubernetes clusters.- - Utilize SnapLogic cloud-native integration platform for designing and implementing integration pipelines. Required Skills : - Expertise in Python frameworks (Django, Flask, FastAPI). - Proficiency in Python libraries (Pandas, NumPy, SQLAlchemy). - Strong experience in designing, developing, and maintaining RESTful APIs. - Familiarity with API security, authentication, and authorization mechanisms (OAuth, JWT). - Good experience and hands-on knowledge of PL/SQL (Packages/Functions/Ref cursors). - Knowledge of data normalization and Oracle performance optimization techniques. - Experience in development & low-level design of warehouse solutions. - Familiarity with Data Warehouse, Datamart and ODS concepts. - Proficiency in AWS services (EC2, S3, RDS, Lambda). Good to Have Skills : Kubernetes : - Hands-on experience with Kubernetes for container orchestration. Infrastructure as Code : - Experience with infrastructure-as-code tools (Terraform, CloudFormation). Integration Platforms : - Experience with SnapLogic cloud-native integration platform. Experience : - 5 to 8 years of experience as a Python Developer. Location : - Bangalore or Gurgaon

We are looking for a Senior Data Engineer to join our team. You will use various methods to transform raw data into useful data systems. You'll strive for efficiency by aligning data systems with business goals. To succeed in this position, you should have strong analytical skills and the ability to combine data from different sources. Data engineer skills also include familiarity with several programming languages and an understanding of machine learning methods. If you are detail-oriented, with excellent organizational skills and experience in this field, wed like to hear from you. Job Requirements Participate in the customer's system design meetings and collect the functional/technical requirements. Responsible to meet the customer expectations on real-time data integrity and implementing efficient solutions A clear understanding of Spark, Hive, Kafka, and RDBMS architecture. Experience in writing Scala/Python programs and SQL queries. Suggest and implement best practices in data integration. Guide the QA team in defining system integration tests as needed. Split the planned deliverables into tasks and assign them to the team. Good to have: Knowledge of CI/CD concepts, Apache Kafka Key traits: Should have excellent communication skills. Should be self-motivated and willing to work as part of a team. Should be able to collaborate and coordinate in a remote environment. Be a problem solver and be proactive to solve the challenges that come his way.

The IT Data Solution Architect plays a critical role in defining, designing, and implementing data architecture solutions that align with business and operational models. Work closely with business stakeholders, IT teams, and data engineers to ensure that data solutions are scalable, secure, and aligned with enterprise goals. Design data flows, specify data integration strategies, and ensure compliance with data governance and security policies. You Have: Knowledge in S4 Hana and/or SAP Commerce Cloud will be plus. Familiarity withLead-To-Cash process and Data migration. Typically requires 4-6 years relevant experience and a graduate equivalent (or higher) degree. It would be nice if you also had: Familiarity with DevOps practices for data engineering, including Git, Jenkins, Docker, and Kubernetes. Awareness of GDPR, CCPA, HIPAA, and other data security and privacy regulations. Collaborates with key stakeholders to analyze ideas and demands for E2E data topics and participates in translating detailed business requirements provided by the solution owner into implementable solution including specific technical requirements. Partners with business and IT on design solution for data requirements. Keeps the alignment between enterprise architecture blueprint, Nokia data governance rules and solution implementation for data requirements across E2E ecosystems. Support and drive necessary meetings and coordination for E2E data readiness for testing cycles (SIT and UAT). Confirm data readiness across E2E ngCore ecosystems.

Research in Nokia Standards (NSD) comprises fundamental systematic research of an exploratory nature for developing new products and services as well as disruptive technologies superseding conventional approaches. It covers the provision of a comprehensive understanding of scientific insights, specifically leading to (pre-) standardization of ideas and concepts for 6G technologies and beyond. In light of the recent advancements in artificial intelligence (AI) and data monetization, 6G will be the first mobile system built around and for datafirst, 6G will be the first generation of fully AI-native mobile systems where new AI features will enhance traditional solutions. Secondly, 6G will be designed according to a data-driven approach, where quantitative data and measures collected in past releases will help in shaping the design of 6G. Thirdly, 6G will be conceived to convert the massive data generated by mobile systems into valuable and monetizableinformation. You must have: Strong technical background in data science and modern data management techniques. Solid knowledge of data analysis techniques to analyze data generated in wireless systems to optimize their performance. Proven experience in data engineering methods to represent data including hands-on experience in developing data management and AI tools. Excellent written and interpersonal communication skills. Leadership and collaborative mindset with the ability to work autonomously. Strong professional ethics, open communication, and ability to rectify mistakes quickly. It would be nice if you also had: Knowledge of language processing techniques (NLP, GenAI, transformers) including hands-on experience of the tools. Experience in simulation of dynamic systems, especially for wireless communication systems (link and/or system level). Hands-on experience with AI/ML techniques applied to wireless communication. Proven software development skills with C/C++ programming language. Proven track record of innovation like publications in top conferences and journals. Drive data science progress in 6G technology and build the next generation mobile systems around modern data management approaches. Drive progress as a technical lead, contributing to new products, processes, and standards on data science for future 6G systems. Work with a world-class team to drive 6G technologies in global standardization. Carry out specialized activities/projects, demonstrating adaptability in complex business environments. Communicate within and outside the team, influencing others to accept new concepts and practices.

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too, Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Must have 5+ years exp in Big Data -Hadoop Spark -Scala ,Python Hbase, Hive Good to have Aws -S3, athena ,Dynomo DB, Lambda, Jenkins GIT Developed Python and pyspark programs for data analysis. Good working experience with python to develop Custom Framework for generating of rules (just like rules engine). Developed Python code to gather the data from HBase and designs the solution to implement using Pyspark. Apache Spark DataFrames/RDD's were used to apply business transformations and utilized Hive Context objects to perform read/write operations, Preferred technical and professional experience Understanding of Devops. Experience in building scalable end-to-end data ingestion and processing solutions Experience with object-oriented and/or functional programming languages, such as Python, Java and Scala

Key Responsibilities Design, develop, and optimize data pipelines using Python and AWS services such asGlue, Lambda, S3, EMR, Redshift, Athena, and Kinesis. Implement ETL/ELT processes to extract, transform, and load data from various sources into centralized repositories (e.g., data lakes or data warehouses). Collaborate with cross-functional teams to understand business requirements and translate them into scalable data solutions. Monitor, troubleshoot, and enhance data workflows for performance and cost optimization. Ensure data quality and consistency by implementing validation and governance practices. Work on data security best practices in compliance with organizational policies and regulations. Automate repetitive data engineering tasks using Python scripts and frameworks. Leverage CI/CD pipelines for deployment of data workflows on AWS. Required Skills and Qualifications Professional Experience:5+ years of experiencein data engineering or a related field. Programming: Strong proficiency inPython, with experience in libraries likepandas,pySpark,orboto3. AWS Expertise: Hands-on experience with core AWS services for data engineering, such as: AWS Gluefor ETL/ELT. S3for storage. RedshiftorAthenafor data warehousing and querying. Lambdafor serverless compute. KinesisorSNS/SQSfor data streaming. IAM Rolesfor security. Databases: Proficiency in SQL and experience withrelational(e.g., PostgreSQL, MySQL) andNoSQL(e.g., DynamoDB) databases. Data Processing: Knowledge of big data frameworks (e.g., Hadoop, Spark) is a plus. DevOps: Familiarity with CI/CD pipelines and tools like Jenkins, Git, and CodePipeline. Version Control: Proficient with Git-based workflows. Problem Solving: Excellent analytical and debugging skills. Optional Skills Knowledge ofdata modelinganddata warehouse designprinciples. Experience withdata visualization tools(e.g., Tableau, Power BI). Familiarity with containerization (e.g., Docker) and orchestration (e.g., Kubernetes). Exposure to other programming languages like Scala or Java.

Data engineers are responsible for building reliable and scalable data infrastructure that enables organizations to derive meaningful insights, make data-driven decisions, and unlock the value of their data assets. - Grade Specific The role support the team in building and maintaining data infrastructure and systems within an organization. Skills (competencies) Ab Initio Agile (Software Development Framework) Apache Hadoop AWS Airflow AWS Athena AWS Code Pipeline AWS EFS AWS EMR AWS Redshift AWS S3 Azure ADLS Gen2 Azure Data Factory Azure Data Lake Storage Azure Databricks Azure Event Hub Azure Stream Analytics Azure Sunapse Bitbucket Change Management Client Centricity Collaboration Continuous Integration and Continuous Delivery (CI/CD) Data Architecture Patterns Data Format Analysis Data Governance Data Modeling Data Validation Data Vault Modeling Database Schema Design Decision-Making DevOps Dimensional Modeling GCP Big Table GCP BigQuery GCP Cloud Storage GCP DataFlow GCP DataProc Git Google Big Tabel Google Data Proc Greenplum HQL IBM Data Stage IBM DB2 Industry Standard Data Modeling (FSLDM) Industry Standard Data Modeling (IBM FSDM)) Influencing Informatica IICS Inmon methodology JavaScript Jenkins Kimball Linux - Redhat Negotiation Netezza NewSQL Oracle Exadata Performance Tuning Perl Platform Update Management Project Management PySpark Python R RDD Optimization SantOs SaS Scala Spark Shell Script Snowflake SPARK SPARK Code Optimization SQL Stakeholder Management Sun Solaris Synapse Talend Teradata Time Management Ubuntu Vendor Management

Role & responsibilities 3 to 4+ years of hands-on experience in SQL database design, data architecture, ETL, Data Warehousing, Data Mart, Data Lake, Big Data, Cloud and Data Governance domains. • Take ownership of the technical aspects of implementing data pipeline & migration requirements, ensuring that the platform is being used to its fullest potential through designing and building applications around business stakeholder needs. • Interface directly with stakeholders to gather requirements and own the automated end-to-end data engineering solutions. • Implement data pipelines to automate the ingestion, transformation, and augmentation of both structured, unstructured, real-time data, and provide best practices for pipeline operations • Troubleshoot and remediate data quality issues raised by pipeline alerts or downstream consumers. Implement Data Governance best practices. • Create and maintain clear documentation on data models/schemas as well as transformation/validation rules. • Implement tools that help data consumers to extract, analyze, and visualize data faster through data pipelines. • Implement data security, privacy, and compliance protocols to ensure safe data handling in line with regulatory requirements. • Optimize data workflows and queries to ensure low latency, high throughput, and cost efficiency. • Leading the entire software lifecycle including hands-on development, code reviews, testing, deployment, and documentation for batch ETL's. • Work directly with our internal product/technical teams to ensure that our technology infrastructure is seamlessly and effectively integrated • Migrate current data applications & pipelines to Cloud leveraging technologies in future Preferred candidate profile • Graduate with Engineering Degree (CS/Electronics/IT) / MCA / MCS or equivalent with substantial data engineering experience. • 3+ years of recent hands-on experience with a modern programming language (Scala, Python, Java) is required; Spark/ Pyspark is preferred. • Experience with configuration management and version control apps (ie: Git) and experience working within a CI/CD framework is a plus. • 3+ years of recent hands-on SQL programming experience in a Big Data environment is required. • Working knowledge of PostgreSQL, RDBMS, NoSQL and columnar databases. • Experience developing and maintaining ETL applications and data pipelines using big data technologies is required; Apache Kafka, Spark, Airflow experience is a must. • Knowledge of API and microservice integration with applications. • Experience with containerization (e.g., Docker) and orchestration (e.g., Kubernetes). • Experience building data solutions for Power BI and Web visualization applications. • Experience with Cloud is a plus. • Experience in managing multiple projects and stakeholders with excellent communication and interpersonal skills. • Ability to develop and organize high-quality documentation. • Superior analytical skills and a strong sense of ownership in your work. • Collaborate with data scientists on several projects. Contribute to development and support of analytics including AI/ML. • Ability to thrive in a fast-paced environment, and to manage multiple, competing priorities simultaneously. • Prior Energy & Utilities industry experience is a big plus. Experience (Min. Max. in yrs.): 3+ years of core/relevant experience Location: Mumbai (Onsite)

Data Science professional with a proven track record in training Engineering, IT, Diploma, Polytechnic and Technical candidates. With over a 7 yrs of experience in Artificial Intelligence, Machine Learning, Big Data, and Cloud Computing, Specialise in delivering industry-oriented, hands-on training that equips candidates with the technical proficiency required in today's data-driven world.

Hybrid - Bengaluru Full time Permanent Position Must have skills required : SQL, product metrics, Data-Driven, Analytical Skills, Hypothesis testing, data-backed goals Good to have skills : Python, B2B SaaS, Data Engineering Read about the release here: Were on a mission to hire the very best and are committed to creating exceptional employee experiences where everyone is respected and has access to equal opportunity. Role Overview: We are seeking a Product Analyst to take ownership of key metrics, reporting, and experimentation within Nanonets. Working directly with our founders, this role involves goal setting, tracking, and prioritizing features based on metrics-driven insights. As a Product Analyst, youll be critical to helping teams set data-informed goals, clarifying the key metrics that drive success, and enabling a hypothesis-driven approach to product development. Key Responsibilities Metrics Ownership: Manage and report on company metrics, setting measurable goals and tracking progress against them. Outcome Definition: Help teams define data-backed goals and establish measurable outcomes, ensuring efforts align with strategic company objectives. Input Metrics Clarity: Identify and communicate the key levers that drive output metrics, providing teams with actionable insights. Hypothesis-Driven Development: Build systems that support hypothesis testing through product features, creating, running, and measuring A/B tests and experiments to validate customer behavior hypotheses. Product Development Optimization: Partner with product teams to establish high-quality goals, focus on the most impactful metrics, and support agile development based on data-driven decision-making. What Were Looking For Education : Bachelors degree preferable from a Tier 1 college. 2- 4 years of relevant experience, such as Data Analyst or Product Analyst, ideally within a B2B SaaS company. Technical Skills : Proficiency in SQL (required). Strong analytical skills and understanding of key product metrics. Willingness to learn Python and enhance data engineering skills. Robust product sense, with the ability to interpret and act on data for feature prioritization. Mindset : Curious, experiment-driven, and eager to tackle complex product challenges through data. Comfortable scrapping or iterating on underperforming features based on data.

We are looking for a Data Engineer with 3 years of expreience that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company. Responsibilities - Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities - Implementing ETL process like importing data etc. - Implementing Data API transformations using Spark and Hadoop cluster Skills and Qualifications - Proficient in Python programming - Working knowledge with LLM's should be good to have Skill set - Proficient understanding of distributed computing principles - Experience with Spark SQL, Python, Spark Data frames. - Experience with integration of data from multiple data sources - Experience with SQL and NoSQL databases such as MySQL, Cassandra, MongoDB - Good understanding of Lambda Architecture/Functional programming, along with its advantages and drawbacks

Key Responsibilities Data Pipeline Development: Design, develop, and optimize robust data pipelines to efficiently collect, process, and store large-scale datasets for AI/ML applications. ETL Processes: Develop and maintain Extract, Transform, and Load (ETL) processes to ensure accurate and timely data delivery for machine learning models. Data Integration: Integrate diverse data sources (structured, unstructured, and semi-structured data) into a unified and scalable data architecture. Data Warehousing & Management: Design and manage data warehouses to store processed and raw data in a highly structured, accessible format for analytics and AI/ML models. AI/ML Model Development: Collaborate with Data Scientists to build, fine-tune, and deploy machine learning models into production environments. Focus on model optimization, scalability, and operationalization. Automation: Implement automation techniques to support model retraining, monitoring, and reporting. Cloud & Distributed Systems: Work with cloud platforms (AWS, Azure, GCP) and distributed systems to store and process data efficiently, ensuring that AI/ML models are scalable and maintainable in the cloud environment. Data Quality & Governance: Implement data quality checks, monitoring, and governance frameworks to ensure the integrity and security of the data being used for AI/ML models. Collaboration: Work cross-functionally with Data Science, Business Intelligence, and other engineering teams to meet organizational data needs and ensure seamless integration with analytics platforms. Required Skills and Qualifications Bachelor's or Masters Degree in Computer Science, Engineering, Data Science, or a related field. Strong proficiency in Python for AI/ML and data engineering tasks. Experience with AI/ML frameworks such as TensorFlow, PyTorch, Scikit-learn, and Keras. Proficient in SQL and working with relational databases (e.g., MySQL, PostgreSQL, SQL Server). Strong experience with ETL pipelines and data wrangling in large datasets. Familiarity with cloud-based data engineering tools and services (e.g., AWS (S3, Lambda, Redshift), Azure, GCP). Solid understanding of big data technologies like Hadoop, Spark, and Kafka for data processing at scale. Experience in managing and processing both structured and unstructured data. Knowledge of version control systems (e.g., Git) and agile development methodologies. Experience with data containers and orchestration tools such as Docker and Kubernetes. Strong communication skills to collaborate effectively with cross-functional teams. Preferred Skills Experience with Data Warehouses (e.g., Amazon Redshift, Google BigQuery, Snowflake). Familiarity with CI/CD pipelines for ML model deployment and automation. Familiarity with machine learning model monitoring and performance optimization. Experience with data visualization tools like Tableau, Power BI, or Plotly. Knowledge of deep learning models and frameworks. DevOps or MLOps experience for automating deployment of models. Advanced statistics or math background for improving model performance and accuracy.

Design and develop robust and scalable ETL processes to ingest and process large datasets from various sources. About Us: We are a dynamic fintech firm dedicated to revolutionizing the financial services industry through innovative data solutions. We believe in leveraging cutting-edge technology to provide superior financial products and services to our clients. Join our team and be a part of this exciting journey. Job Overview: We are looking for a skilled Data Engineer with 3-5 years of experience to join our data team. The ideal candidate will have a strong background in ETL processes, data pipeline creation, and database management. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data systems and pipelines. Key Responsibilities: Design and develop robust and scalable ETL processes to ingest and process large datasets from various sources. Build and maintain efficient data pipelines to support real-time and batch data processing. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions. Optimize database performance and ensure data integrity and security. Troubleshoot and resolve data-related issues and provide support for data operations. Implement data quality checks and monitor data pipeline performance. Document technical solutions and processes for future reference. Required Skills and Qualifications: Bachelor's degree in Engineering, or a related field. 5-8 years of experience in data engineering or a related role. Strong proficiency in ETL tools and techniques. Experience with SQL and relational databases (e.g., MySQL, PostgreSQL). Familiarity with big data technologies Proficiency in programming languages such as Python, Java, or Scala. Knowledge of data warehousing concepts and tools Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills. Preferred Qualifications: Experience with data visualization tools (e.g., Tableau, Power BI). Knowledge of machine learning and data science principles. Experience with real-time data processing and streaming platforms (e.g., Kafka). What We Offer: Competitive compensation package (12-20 LPA) based on experience and qualifications. Opportunity to work with a talented and innovative team in the fintech industry.. Professional development and growth opportunities. Knowledge of data warehousing concepts and tools

Development: Design, build, and maintain robust, scalable, and high-performance data pipelines to ingest, process, and store large volumes of structured and unstructured data. Utilize Apache Spark within Databricks to process big data efficiently, leveraging distributed computing to process large datasets in parallel. Integrate data from a variety of internal and external sources, including databases, APIs, cloud storage, and real-time streaming data. Data Integration & Storage: Implement and maintain data lakes and warehouses, using technologies like Databricks, Azure Synapse, Redshift, BigQuery to store and retrieve data. Design and implement data models, schemas, and architecture for efficient querying and storage. Data Transformation & Optimization: Leverage Databricks and Apache Spark to perform data transformations at scale, ensuring data is cleaned, transformed, and optimized for analytics. Write and optimize Spark SQL, PySpark, and Scala code to process large datasets in real-time and batch jobs. Work on ETL processes to extract, transform, and load data from various sources into cloud-based data environments. Big Data Tools & Technologies: Utilize cloud-based big data platforms (e.g., AWS, Azure, Google Cloud) in conjunction with Databricks for distributed data processing and storage. Implement and maintain data pipelines using Apache Kafka, Apache Flink, and other data streaming technologies for real-time data processing. Collaboration & Stakeholder Engagement: Work with data scientists, data analysts, and business stakeholders to define data requirements and deliver solutions that align with business objectives. Collaborate with cloud engineers, data architects, and other teams to ensure smooth integration and data flow between systems. Monitoring & Automation: Build and implement monitoring solutions for data pipelines, ensuring consistent performance, identifying issues, and optimizing workflows. Automate data ingestion, transformation, and validation processes to reduce manual intervention and increase efficiency. Document data pipeline processes, architectures, and data models to ensure clarity and maintainability. Adhere to best practices in data engineering, software development, version control, and code review. Required Skills & Qualifications: Education: Bachelors degree in Computer Science, Engineering, Data Science, or a related field (or equivalent experience). Technical Skills: Apache Spark: Strong hands-on experience with Spark, specifically within Databricks (PySpark, Scala, Spark SQL). Experience working with cloud-based platforms such as AWS, Azure, or Google Cloud, particularly in the context of big data processing and storage. Proficiency in SQL and experience with cloud data warehouses (e.g., Redshift, BigQuery, Snowflake). Strong programming skills in Python, Scala, or Java. Big Data & Cloud Technologies: Experience with distributed computing concepts and scalable data processing architectures. Familiarity with data lake architectures and frameworks (e.g., AWS S3, Azure Data Lake). Data Engineering Concepts: Strong understanding of ETL processes, data modeling, and database design. Experience with batch and real-time data processing techniques. Familiarity with data quality, data governance, and privacy regulations. Problem Solving & Analytical Skills: Strong troubleshooting skills for resolving issues in data pipelines and performance optimization. Ability to work with large, complex datasets, and perform data wrangling and cleaning.

We are seeking a highly skilled and experienced Senior Data Engineer to join our team. This role will be instrumental in designing, developing, and maintaining our data infrastructure, ensuring the effective processing and analysis of large datasets. The ideal candidate will have a strong background in data modeling, data architecture, and experience with a variety of data technologies. Key Responsibilities: Design and implement robust data pipelines, ETL processes, and data warehouses to support our analytics and reporting needs. Develop and maintain data models, schemas, and metadata to ensure data quality and consistency. Collaborate with data scientists, analysts, and business stakeholders to understand their requirements and translate them into technical solutions. Optimize data pipelines for performance and scalability to handle large volumes of data. Stay up-to-date with the latest data technologies and trends to drive innovation and efficiency. Responsibilities Design, develop, and maintain scalable data architectures, pipelines, APIs, and integrations. Create and optimize data models to support efficient data processing and storage. Manage and maintain databases, including Postgres, SSIS, ensuring data integrity and performance. Develop, deploy, and manage ETL and EDI processes. Develop and maintain scripts and applications using Python for data processing and analysis. Ensure data security and compliance with relevant regulations and best practices. Leverage cloud services (e.g., Azure, AWS) for data storage, processing, and analytics. Collaborate with cross-functional teams to gather requirements and provide data-driven insights. Implement and manage caching solutions to improve data retrieval speeds. Create and maintain comprehensive documentation for all data processes and architectures. Utilize data visualization tools to create interactive dashboards and reports for stakeholders. Qualifications Bachelors degree in Computer Science, Information Systems, Data Science, or a related field (or equivalent work experience). Minimum of 5 years of experience in data engineering or a related field. Proficiency in data modeling, data architecture, and database management with Postgres or SSIS.. Experience with electronic medical records (EMRs) and understanding of the healthcare industry is strongly desired. Strong SQL skills and experience with common ETL tools. Proficiency in Python for data processing and automation. Experience with common caching solutions (e.g., Redis, Memcached). Expertise in data security best practices and regulatory compliance. Hands-on experience with cloud platforms like Azure and AWS. Proficiency with data visualization tools such as Power BI, Tableau, or similar. Excellent problem-solving skills and ability to troubleshoot data issues effectively. Strong communication skills, both written and verbal, with the ability to explain complex technical concepts to non-technical stakeholders. Desired Skills Knowledge of data warehousing concepts and methodologies. Experience with Agile/Scrum methodologies. Familiarity with Power BI administration and deployment.