855 Data Ingestion Jobs - Page 4

About The Role Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Snowflake Data Warehouse Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform components. Your typical day will involve collaborating with Integration Arch...

About The Role Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Snowflake Data Warehouse Good to have skills : NA Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform components. Your typical day will involve collaborating with Integration Arc...

Role description We are seeking talented Technical Program Manager?role specifically tailored for working with the?Adobe Experience Platform (AEP) to join our India delivery team. Role Overview As a ? Technical Program Manager (TPM) , y ou will be responsible for leading and delivering complex technology projects related to Adobe Experience Platform . This role requires deep technical expertise , strong program management skills, and ability to collaborate across cross-functional teams to align technical initiatives with business goals. Key Responsibilities Drive end-to-end planning and execution of technical programs across Adobe Experience Platform, including data ingestion, identity resol...

At EY, youll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And were counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. PositionSenior - Data Engineer Role Description: Leads the delivery of solution or infrastructure development services for a large or more complex project, using advanced technical capabilities Takes accountability for the design, development, delivery and maintenance of solutions or infrastructure, driving compliance with and contributing to t...

Job TitleAWS Senior Data Engineer Experience RequiredMinimum 3-7 years Job Summary: We are seeking a skilled Data Engineer with a strong background in data ingestion, processing, and storage. The ideal candidate will have experience working with various data sources and technologies, particularly in a cloud environment. You will be responsible for designing and implementing data pipelines, ensuring data quality, and optimizing data storage solutions. Key Responsibilities: Design, develop, and maintain scalable data pipelines for data ingestion and processing using Python, Spark, and AWS services. Work with on-prem Oracle databases, batch files, and Confluent Kafka for data sourcing. Implemen...

About The Role ::Tester & Test Lead - ETL, Data Data testing experience mandatory. SQL proficiency in Big Query is Mandatory. Experience in GCP DataForm, Great Expectation,DBT is preferable. Experience in testing data validation scenarios and Data ingestion, pipelines, and transformation processes. Demonstrated experience of hands-on Issue and Defect Management Experience of Agile Software Development & Testing methods for deployment in cloud environments 4 to 6 years of Testing experience for Test Engineer and 6+ years of Testing experience for Test Lead profiles.

Experienced data modelers, SQL, ETL, with some development background to provide defining new data schemas, data ingestion for Adobe Experience Platform customers. Interface directly with enterprise customers and collaborate with internal teams. 10+ years of strong experience with data transformation & ETL on large data sets/5+ years of Data Modelling experience (i.e., Relational, Dimensional, Columnar, Big Data) 5+ years of complex SQL or NoSQL experience Experience in advanced Data Warehouse concepts

Title and Summary MLOps Engineering DirectorThe Security Solutions Data Science team is responsible for creating Artificial Intelligence (AI) and Machine Learning (ML) models backing its flagship product The models generated are production ready and created to back specific products in Mastercards authentication and authorization networks The Data Science team is also responsible for developing automated processes for creating models covering all modeling steps, from data extraction up to delivery In addition, the processes must be designed to scale, to be repeatable, resilient, and industrialized Services within Mastercard is responsible for acquiring, engaging, and retaining customers by m...

As a data engineer at Verizon, you will be part of a world-class team dedicated to driving the telecom business to its full potential. You will play a crucial role in building data products for telecom wireless and wireline business, including consumer analytics, telecom network performance, and service assurance analytics. Working with cutting-edge technologies like digital twin, you will collaborate with business product owners, data scientists, and system architects to develop strategic data solutions from various sources. Your responsibilities will include data ingestion, preparation, transformation, developing data streaming applications, ETL/ELT development, and contributing to DevOps ...

Job Purpose and Impact The Professional, Data Engineering job designs, builds and maintains moderately complex data systems that enable data analysis and reporting. With limited supervision, this job collaborates to ensure that large sets of data are efficiently processed and made accessible for decision making. Key Accountabilities DATA & ANALYTICAL SOLUTIONS : Develops moderately complex data products and solutions using advanced data engineering and cloud based technologies, ensuring they are designed and built to be scalable, sustainable and robust. DATA PIPELINES: Maintains and supports the development of streaming and batch data pipelines that facilitate the seamless ingestion of data ...

Project description Enterprise Technology Infrastructure, Platforms & Services (ETIPS) is leading the modernization of UnitedHealth Group legacy and on-prem landscape with a thoughtful migration to the cloud to help create more flexibility for speed and stability for end users. As the owner of ETIPS technology portfolio, we are exploring new ways of delivering these investments and seeking a strategic partnership to co-deliver and accelerate our ambitious and innovative agenda. Vendor will provide technology services for Core Technology components. Responsibilities Design, develop, and operate secure cloud platform services and software that meet business requirements. Lead a team of cloud p...

Develop and maintain data pipelines, ELT processes, and workflow orchestration using Apache Airflow, Python and PySpark to ensure the efficient and reliable delivery of data. Design and implement custom connectors to facilitate the ingestion of diverse data sources into our platform, including structured and unstructured data from various document formats . Collaborate closely with cross-functional teams to gather requirements, understand data needs, and translate them into technical solutions. Implement DataOps principles and best practices to ensure robust data operations and efficient data delivery. Design and implement data CI/CD pipelines to enable automated and efficient data integrati...

Skill : Big Data Need to Have : Python, Pyspark, Trino, Hive Good to Have : Snowflake, SQL, Airflow, Openshift, Kubernentes Location : Hyderabad, Pune, Bangalore Job Description : Develop and maintain data pipelines, ELT processes, and workflow orchestration using Apache Airflow, Python and PySpark to ensure the efficient and reliable delivery of data. Design and implement custom connectors to facilitate the ingestion of diverse data sources into our platform, including structured and unstructured data from various document formats . Collaborate closely with cross functional teams to gather requirements, understand data needs, and translate them into technical solutions. Implement DataOps pr...

Experienced data modelers, SQL, ETL, with some development background to provide defining new data schemas, data ingestion for Adobe Experience Platform customers. Interface directly with enterprise customers and collaborate with internal teams. 10+ years of strong experience with data transformation & ETL on large data sets/5+ years of Data Modelling experience (i.e., Relational, Dimensional, Columnar, Big Data) 5+ years of complex SQL or NoSQL experience Experience in advanced Data Warehouse concepts.

JD GCP Data Engineer Responsibilities: Design, develop, and maintain data pipelines using GCP services such as Dataflow, BigQuery, and Pub/Sub. Implement and manage data warehouse solutions using BigQuery, ensuring data is stored securely and efficiently. Use Pub/Sub for real time data ingestion and streaming analytics. Provision and manage GCP infrastructure using Terraform, ensuring best practices in IaC are followed. Optimize data storage and retrieval processes to enhance performance and reduce costs. Monitor and troubleshoot data pipeline issues, ensuring high availability and reliability of data services. Ensure data quality and integrity through robust testing and validation processes...

As a Technical Project Manager at Arabelle Solutions, you will be responsible for the wing-to-wing execution of technical projects for Data & Analytics (D&A) to deliver high-quality outcomes to internal and/or external customers. You will collaborate with business and functional partners and technology leadership to specify requirements and drive technology discussions and strategies in line with business needs. Your role will involve defining execution plans and approaches based on project scope, expected timelines, and available budget/resources. You will also facilitate the convergence of functional and technical knowledge and work with a technical team to coordinate day-to-day execution ...

As a candidate for this role, you would be responsible for the following: - Understanding of Spark core concepts like RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming. - Experience with Spark optimization techniques. - Deep knowledge of Delta Lake features like time travel, schema evolution, data partitioning. - Ability to design and implement data pipelines using Spark and Delta Lake as the data storage layer. - Proficiency in Python/Scala/Java for Spark development and integrate with ETL process. - Knowledge of data ingestion techniques from various sources (flat files, CSV, API, database). - Understanding of data quality best practices and data validation techniques. Qualificatio...

As an experienced candidate with over 8 years of overall experience and 5+ years of relevant experience, your role will involve the following key responsibilities: - Strong understanding of Spark core including RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming - Proficiency in Delta Lake features such as time travel, schema evolution, and data partitioning - Designing and building data pipelines using Spark and Delta Lake - Solid experience in Python/Scala/Java for Spark development - Knowledge of data ingestion from files, APIs, and databases - Familiarity with data validation and quality best practices - Working knowledge of data warehouse concepts and data modeling - Hands-on expe...

Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Job Role Key skills Azure Data Factory (primary) , Azure Data bricks Spark (PySpark, SQL) Must-have skills Cloud certified in one of these categories Azure Data Engineer Azure Data Factory , Azure Data bricks Spark (PySpark or scala), SQL, DATA Ingestion, Curation Semantic Modelling/ Optimization of data model to work within Rahona Experience in Azure ingestion from on-prem source, e.g. mainframe, SQL server, Oracle. Experience in Sqoop / Hadoop Microsoft Excel (for me...

Role & responsibilities Core skills Deep OpenSearch knowledge: In-depth understanding of OpenSearch concepts, including search algorithms, indexing, query optimization, and cluster management. AWS ecosystem: Strong proficiency in relevant AWS services, such as IAM, VPC, S3, CloudWatch, and Kinesis, which integrate with OpenSearch Service. Data ingestion and pipelines: Experience with data ingestion and transformation tools like Logstash, Fluentd, or OpenSearch Ingestion. Troubleshooting and monitoring: The ability to monitor OpenSearch clusters for performance, troubleshoot issues, and set up effective alerting. Networking: Knowledge of network access controls, VPCs, and endpoint configurati...

Project description Public Cloud engineering team is responsible for developing software, managing security and financial controls, and creating reusable application patterns that enable the enterprise to operate efficiently within the public cloud space. The product teams leverage this platform to deliver value in a safe, quick, and sustainable manner, aligning with mission to help people live healthier lives and makes the health system work better for everyone. Join our team and work collaboratively with product owners, operational staff, and technology leadership in an agile environment to create business value. Your contributions will be utilized by thousands engineers as they develop ma...

About The Role Project Role : Security Architect Project Role Description : Define the cloud security framework and architecture, ensuring it meets the business requirements and performance goals. Document the implementation of the cloud security controls and transition to cloud security-managed operations. Must have skills : Security Information and Event Management (SIEM) Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Security Architect, you will define the cloud security framework and architecture, ensuring it meets the business requirements and performance goals. Your typical day will involve col...

Job Description About the Position We have an excellent opportunity available for an Associate Data Engineer to make a difference at Kraft Heinz The Associate Data Engineer role lies within the newly created Analytics Centre of Excellence within the Global Business Services organization The Analytics COE has been set up to build, maintain and enhance analytical products for our business to generate actionable insights from data and make better informed decisions The individual for this position will be a technical engineer in the areas of data management, data quality, data governance, and data engineering They will provide hands-on design, preparation, and development of data solutions for ...

Salary: 15 to 30 LPA Exp: 3 to 8 years Location : Gurgaon/Bangalore/Pune/Chennai Notice: Immediate to 30 days..!! Key Responsibilities & Skillsets: Common Skillsets : 3+ years of experience in analytics, Scala, Pyspark, Python, Spark, SQL and associated data engineering jobs. Must have experience with managing and transforming big data sets using pyspark, spark-scala, Numpy pandas Excellent communication & presentation skills Experience in managing Python codes and collaborating with customer on model evolution Good knowledge of data base management and Hadoop/Spark, SQL, HIVE, Python (expertise). Superior analytical and problem solving skills Should be able to work on a problem independentl...

skilled and motivated Data Engineer with hands-on experience in cloud platforms like AWS, Azure, or GCP to join our growing data team. have strong knowledge of data architecture, pipeline development, and cloud-based data processing.