10328 Data Engineering Jobs - Page 45

Databricks Developer Must-have skills Cloud certified in one of these categories Azure Data Engineer Azure Data Factory , Azure Data bricks Spark (PySpark or scala), SQL, DATA Ingestion, Curation . Semantic Modelling/ Optimization of data model to work within Rahona Experience in Azure ingestion from on-prem source, e.g. mainframe, SQL server, Oracle. Experience in Sqoop / Hadoop Microsoft Excel (for metadata files with requirements for ingestion) Any other certificate in Azure/AWS/GCP and data engineering hands-on experience in cloud Strong Programming skills with at least one of Python, Scala or Java Location- Pune, Bangalore, Coimbatore, Chennai Mandatory Skills: DataBricks - Data Enginee...

Job Description Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do 1. Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information needs, studying systems flow, data usage and work processes Investigating problem areas followed by the software development life cycle Facilitate root cause analysis of the system issues and problem statement Identify ideas to improve system performance and impact availability Analyze client requirements and convert re...

Role Purpose The purpose of the role is to define, architect and lead delivery of machine learning and AI solutions Do 1. Demand generation through support in Solution development a. Support Go-To-Market strategy i. Collaborate with sales, pre-sales &consulting team to assist in creating solutions and propositions for proactive demand generation ii. Contribute to development solutions, proof of concepts aligned to key offerings to enable solution led sales b. Collaborate with different colleges and institutes for recruitment, joint research initiatives and provide data science courses 2. Revenue generation through Building & operationalizing Machine Learning, Deep Learning solutions a. Devel...

Job Description Role Purpose The purpose of the role is to create exceptional architectural solution design and thought leadership and enable delivery teams to provide exceptional client engagement and satisfaction. Do 1.Develop architectural solutions for the new deals/ major change requests in existing deals Creates an enterprise-wide architecture that ensures systems are scalable, reliable, and manageable. Provide solutioning of RFPs received from clients and ensure overall design assurance Develop a direction to manage the portfolio of to-be-solutions including systems, shared infrastructure services, applications in order to better match business outcome objectives Analyse technology en...

Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do 1. Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information needs, studying systems flow, data usage and work processes Investigating problem areas followed by the software development life cycle Facilitate root cause analysis of the system issues and problem statement Identify ideas to improve system performance and impact availability Analyze client requirements and convert requirements to fe...

Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do 1. Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information needs, studying systems flow, data usage and work processes Investigating problem areas followed by the software development life cycle Facilitate root cause analysis of the system issues and problem statement Identify ideas to improve system performance and impact availability Analyze client requirements and convert requirements to fe...

Job Overview : We are looking for a skilled Full Stack Developer who thrives at the intersection of software development, modern data infrastructure, and internal tools. Youll help architect and build robust APIs, manage databases like PostgreSQL and Supabase, and contribute to the frontend when needed all while enabling automation and actionable business insights. This role combines backend craftsmanship with data intuition to power internal tools, data reporting pipelines, and intelligent automation systems. Key Responsibilities : Application & API Development: Design and maintain scalable backend services using Python (Django). Build and optimize RESTful or GraphQL APIs for internal and e...

Company Description Blendis a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. With a mission to fuel bold visions, Blend tackles significant challenges by seamlessly aligning human expertise with artificial intelligence. The company is dedicated to unlocking value and fostering innovation for its clients by harnessing world-class people and data-driven strategy. We believe that the power of people and AI can have a meaningful impact on your world, creating more fulfilling work and projects for our people and clients. For more information, visit www.blend360.com Job Description We are seeki...

Role Description As a Senior Technical Lead - Data Science and Modeling at Incedo, you will be responsible for developing and deploying predictive models and machine learning algorithms to support business decision-making. You will work with data scientists, data engineers, and business analysts to understand business requirements and develop data-driven solutions. You will be skilled in programming languages such as Python or R and have experience in data science tools such as TensorFlow or Keras. You will be responsible for ensuring that models are accurate, efficient, and scalable. Roles & Responsibilities: Developing and implementing machine learning models and algorithms to solve comple...

Role & responsibilities Location: Remote Notice Period : Immediate Mandatory Skills : Azure , Data Engineer , Sql Server ,Python and R. Review and optimisation of our current SQL Database (tables, views) extensive experience in Data Warehousing / Data Engineering on SQL Server Enhancing the documentation of our data warehouse (data dictionary, logical data model) Review of the ETL scripts (developed in Python and/or R) Migration of R scripts to Python along with improvements Establishing guidelines for better structuring our scripts and documenting them properly Implementation of data quality checks, logging and alerting Recommended ways of anonymising some historical data Assessment of usin...

Overview Job Summary : We are looking for a skilled GCP Data Engineer with 3 to 6 years of experience to join our dynamic team. The ideal candidate will have a strong background in data engineering, cloud technologies, and experience with Google Cloud Platform (GCP) services. You will be responsible for designing, building, and maintaining scalable data pipelines and frameworks that support our data analytics and machine learning initiatives. Key Responsibilities : Design and implement data pipelines using GCP services such as BigQuery, Dataflow, and Pub/Sub. Collaborate with data scientists and analysts to understand data needs and deliver appropriate solutions. Optimize data storage and pr...

Job Summary: We are looking for a skilled GCP Data Engineer to design, build, and optimize data pipelines and solutions on Google Cloud Platform. The ideal candidate will have hands-on experience with BigQuery, Composer, Dataproc, Python, PySpark, Airflow, and SQL , and will play a key role in ensuring data reliability, scalability, and efficiency across our data ecosystem. Collaborate with data analysts, scientists, and other stakeholders to deliver reliable data solutions. Monitor and troubleshoot data pipelines and ensure data accuracy and consistency. Optimize cloud resources for cost efficiency and performance . Maintain clear technical documentation for all data processes and workflows...

Responsibilities include: Mandatory Skills: Snowflake experience, Data Architecture experience, ETL process experience, Large Data migration solutioning experience Lead the design and architecture of data solutions leveraging Snowflake, ensuring scalability, performance, and reliability. Oversee the implementation of data pipelines, ETL processes, and data governance frameworks within Snowflake environments. Ensure efficient data extraction from SAP BW/ECC systems Collaborate with stakeholders to understand business requirements and translate them into technical specifications and data models. Develop and maintain data architecture standards, guidelines, and best practices, including data go...

Overview Prodapt is lookig for Lead data engineers having 5+ years of experience having hands on expertise on GCP, BQ, Data Pipeline creation & ETL. Responsibilities Understanding the requirements Performing in-depth analysis on data Setting up secure data pipelines from ingestion to delivery Individual contributor Requirements 5- 10 years of experience in Data Engineering with strong ability to work independently, communicate effectively, and deliver high-quality solutions. Should have strong SQL programming skills. Experience in cloud-based systems (GCP, BQ) is must. Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues. Expertise i...

Overview Join the Prodapt team in building a unified, enterprise-scale solution for model, data, and service management across the AI/ML model development. Responsibilities Proficiency in Python and experience with pyspark and ML/AI model development (TensorFlow, scikit-learn, MLflow). Experience in building applications with Java, including the use of Spring for backend development. Hands-on experience with core GCP services, including GKE, Dataproc, BigQuery, and Google Cloud Storage (GCS). Strong understanding of data storage and management (Oracle Exadata, HDFS, Hive, GCS, key-value stores). Experience with CI/CD tools and automation for ML/data workflows. Knowledge of security best prac...

Key Responsibilities: Optimize scalable data pipelines for ingestion, transformation, and loading of large datasets from diverse sources. Utilize Azure Data Factory for orchestration and automation of ETL/ELT processes, ensuring efficient data flow. Work extensively with Azure services such as Azure Blob Storage and Azure Data Lake Storage (ADLS Gen2) for efficient data storage and management. Integrate and manage data flows using Azure Logic Apps for event-driven automation and workflow orchestration. Develop and maintain robust SQL queries, stored procedures, and data models to support reporting and analytical requirements. Implement and enforce data governance, quality, and security best ...

Skills- AWS Data Engineer Experience: 9 to 13 years Location: AIA-Pune We are seeking skilled and dynamic Cloud Data Engineers specializing in AWS, Databricks. The ideal candidate will have a strong background in data engineering, with a focus on data ingestion, transformation, and warehousing. They should also possess excellent knowledge of PySpark or Spark, and a proven ability to optimize performance in Spark job executions. Key Responsibilities: - Design, build, and maintain scalable data pipelines for a variety of cloud platforms including AWS. - Implement data ingestion and transformation processes to facilitate efficient data warehousing. - Utilize cloud services to enhance data proce...

Your role and responsibilities As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need...

Your Role Excellent Communication Skills. Experience working with Fact and Dimension tables, SCD. Proven analytical skills and Problem-solving attitude. Any other GCP services or databases like Datastore, Bigtable, Spanner, Cloud Run, Cloud Functions etc. Should have worked on handling big data. experience in Agile methodologies ETL, ELT skills, Data movement skills, Data processing skills. Certification on Professional Google Cloud Data engineer will be an added advantage. Ability to effectively function in a cross-teams environment Your Profile Require 5+ years experience and relevant is minimum 3 years' experience in GCP Data Engineering. Strong experience in GCP Services, data engineerin...

Cloud Platforms ( AWS, MS Azure, GC etc.)Containerization and Orchestration ( Docker, Kubernetes etc..)APIs - Change APIs to APIs developmentData Pipeline construction using languages like Python, PySpark, and SQLData Streaming (Kafka and Azure Event Hub etc..)Data Parsing ( Akka and MinIO etc..)Database Management ( SQL and NoSQL, including Clickhouse, PostgreSQL etc..)Agile Methodology ( Git, Jenkins, or Azure DevOps etc..)JS like Connectors/ framework for frontend/backendCollaboration and Communication

Your role and responsibilities As Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. Youll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, youll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, youll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include * Implementing and validating predictive models as well as creating and maintain statistical models with a fo...

Your role and responsibilities Develop, test and support future-ready data solutions for customers across industry verticals Develop, test, and support end-to-end batch and near real-time data flows/pipelines Demonstrate understanding in data architectures, modern data platforms, big data, analytics, cloud platforms, data governance and information management and associated technologies Communicates risks and ensures understanding of these risks. Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Minimum of 15+ years of related experience required Experience in Production grade Power BI dashboards with optimized data modelin...

Your role and responsibilities * Analyze existing reports and dashboards built in IBM Cognos and design equivalent solutions in Qlik Sense. * Work closely with business stakeholders to understand reporting requirements and ensure seamless transition. * Optimize Qlik Sense reports for performance, scalability, and user experience. * Provide technical guidance to junior developers and support them on best practices in performance tuning and visual design. * Ensure data accuracy, consistency, and quality across migrated reports. * Collaborate with data engineering teams to ensure proper data sourcing and modeling. * Leverage knowledge of Power BI and Tableau to support cross-platform initiative...

Role & responsibilities Good in Snowflake core, knowledge on Cortex, ADF and Data Mesh Concepts Snowflake core, ADF and Data Mesh Concepts Architect, design, develop,and support Snowflake servicesand warehouse Educational qualification : Bachelors in Tech/ Masters in Tech Locations : Pune/Chennai/Bangalore/Hyderabad - Hybrid

Must to Have Skill Python, Apache (Beem developer) templates to be built by scratch, knowledge on cloud composed and SQL Good to Have Skill Healthcare projects and GCP Healthcare API