855 Data Ingestion Jobs - Page 7

The D. E. Shaw group is a global investment and technology development firm with more than $60 billion in investment capital as of March 1, 2024, and offices in North America, Europe, and Asia. Since our founding in 1988, our firm has earned an international reputation for successful investing based on innovation, careful risk management, and the quality and depth of our staff. We have a significant presence in the world's capital markets, investing in a wide range of companies and financial instruments in both developed and developing economies. We seek an experienced data engineer to join our Equities team at the D. E. Shaw India office in Hyderabad/Gurugram. The Equities India group colla...

Horizontal Data Science Enablement Team within SSO Data Science is looking for a Senior MLOps Engineer who can help solve MLOps problems, manage the Databricks platform for the entire organization, build CI/CD or automation pipelines, and lead best practices All about you : Assist in the administration, configuration, and maintenance of Databricks clusters and workspaces Monitor Databricks clusters for high workloads or excessive usage costs, and promptly alert relevant stakeholders to address issues impacting overall cluster health Implement and manage security protocols, including access controls and data encryption, to safeguard sensitive information in adherence with Mastercard standards...

Do you want to work on innovative projects, collaborate with a dynamic and supportive team, and receive investment in your professional development? At DTCC, we are at the forefront of innovation in the financial markets. We are committed to helping our employees grow and succeed. We believe that you have the skills and drive to make a real impact. We foster a thriving internal community and are committed to creating a workplace that looks like the world that we serve. DTCC Digital Assets is at the forefront of driving institutional adoption of digital assets technology with a steadfast commitment to innovation anchored in security and stability. As the financial services industrys trusted t...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing...

Adobe Campaign & Data Analytics with Cloud Migration Expertise Job Summary Synechron seeks a highly skilled and proactive Associate Specialist with experience in Adobe Campaign and related digital marketing technologies. The role is central to designing, implementing, and maintaining marketing campaign tracking solutions, as well as building analytics dashboards to support marketing and finance teams. You will work closely with cross-functional teams within an agile environment to develop scalable data processing solutions, automate workflows, and ensure seamless deployment and support of marketing applications across testing, staging, and production environments. Your expertise will directl...

Primary Title : Senior LLM Engineer (4+ years) - Hybrid, India. About The Opportunity : A technology consulting firm operating at the intersection of Enterprise AI, Generative AI and Cloud Engineering seeks an experienced LLM-focused engineer. You will build and productionize LLM-powered products and integrations for enterprise customers across knowledge management, search, automation, and conversational AI use-cases. This is a hybrid role based in India for candidates with strong hands-on LLM engineering experience. Role & Responsibilities : - Own design and implementation of end-to-end LLM solutions : data ingestion / retrieval (RAG) / fine-tuning / inference and monitoring for production ...

Primary Title : Senior LLM Engineer (4+ years) - Hybrid, India. About The Opportunity : A technology consulting firm operating at the intersection of Enterprise AI, Generative AI and Cloud Engineering seeks an experienced LLM-focused engineer. You will build and productionize LLM-powered products and integrations for enterprise customers across knowledge management, search, automation, and conversational AI use-cases. This is a hybrid role based in India for candidates with strong hands-on LLM engineering experience. Role & Responsibilities : - Own design and implementation of end-to-end LLM solutions : data ingestion / retrieval (RAG) / fine-tuning / inference and monitoring for production ...

As a Data Engineer at Tide, you will be responsible for building and running data pipelines and services to support business functions, reports, and dashboards. You will work with technologies such as BigQuery/Snowflake, Airflow, Stitch/Fivetran, dbt, Tableau/Looker to enhance our business intelligence capabilities. Your role involves developing end-to-end ETL/ELT pipelines, mentoring junior engineers, troubleshooting technical issues, and improving data pipeline delivery. - Developing end to end ETL/ELT Pipeline in collaboration with Data Analysts - Designing, developing, and implementing scalable, automated processes for data extraction and analysis - Mentoring Junior Engineers in the Team...

Role Overview: As an ETL Development & Testing Analyst in Pune (WFO), your primary responsibility is to ensure a Quality First culture within the team. You will be involved in various tasks related to ETL pipelines, data migration, data ingestion, and more. Key Responsibilities: - Experience in ETL pipelines, data migration, data ingestion, etc. - Strong hands-on experience in Azure, Azure Data Factory, SQL. - Strong hands-on experience in Python scripting. - Experience of developing and executing Test automation using SQL, Python, etc. - Experience with Agile methodology, Jira, and Test management tools. Qualifications Required: - Skills in agile, Jira, data ingestion, testing, SQL, Python,...

As a Data Engineer specializing in Spark & Delta Lake, your role will involve understanding key Spark concepts such as RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming. You will also be expected to have experience in optimizing Spark performance and a deep understanding of Delta Lake features like time travel, schema evolution, and data partitioning. Your responsibilities will include designing and implementing data pipelines using Spark and Delta Lake as the data storage layer. Proficiency in Python, Scala, or Java for Spark development and integration with ETL processes is essential. Additionally, you should possess knowledge of data ingestion techniques from various sources such ...

As a Data Science Solution Architect at Tredence, you will lead the design and development of data science solutions that leverage advanced analytics, machine learning, and artificial intelligence to address complex business challenges. You will work closely with cross-functional teams to understand business requirements and translate them into scalable and efficient data solutions. Key Responsibilities: - Architect end-to-end data science solutions, including data ingestion, processing, modeling, and deployment, to address specific business problems. - Provide technical leadership and guidance to data scientists and engineers, ensuring alignment with best practices and industry standards. -...

Skill- Salesforce Data Cloud Experience: 4 to 7 Years Location - Pune, Bangalore, Mumbai, Gurgaon Immediate Joiners Skills required - Proficiency in Salesforce Data Cloud features and architecture. - Strong understanding of data modelling, data transformation, and ETL processes. - Experience in SQL, data integration tools, and Salesforce connectors. Technical Skills - Implement and manage data ingestion, organization, and activation processes in Salesforce DataCloud. - Build and optimize data models and connections across Salesforce and external systems. - Design and implement data flows, match-and-merge processes, and data identity resolutions. - Analyze and resolve data-related issues to m...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Company Overview Domo's AI and Data Products Platform lets people channel AI and data into innovative uses that deliver a measurable impact. Anyone can use Domo to prepare, analyze, visualize, automate, and build data products that are amplified by AI. Position Summary Working as a member of Domo's Client Services team, the Technical Consultant will be focused on the implementation of fault tolerant, highly scalable solutions. The successful candidate will have a minimum of 5 years working hands-on with data. This individual will join an enthusiastic, fast-paced and dynamic team at Domo. A successful candidate will have demonstrated sustained exceptional performance, innovation, creativity, ...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelo...

Job Summary: Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders. Responsibilities: Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes. Create the data integration and data diagram documentation. Lead the data validation, UAT and regression test for new data asset creation. Create and maintain data models, including schema design and optimization. Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency. Qualifications and Skills: Strong knowledge on Python and Pyspark Expe...

Backend Developer Location : Chennai, India Experience : 10-12 years Your Role: As a Backend Developer - Full-stack Developer, you will be instrumental in designing and building robust backend systems. Youll work closely with cross-functional teams to develop scalable APIs, integrate cloud services, and build data pipelines that support high-performance maritime applications. In this role, you will: Design and develop RESTful and GraphQL APIs Build and maintain scalable data ingestion and processing pipelines Integrate AWS services such as Lambda, API Gateway, and others into backend workflows Collaborate with frontend developers, architects, and DevOps engineers to ensure seamless system in...

About The Job Role Works in the area of Software Engineering, which encompasses the development, maintenance and optimization of software solutions/applications. 1. Applies scientific methods to analyse and solve software engineering problems. 2. He/she is responsible for the development and application of software engineering practice and knowledge, in research, design, development and maintenance. 3. His/her work requires the exercise of original thought and judgement and the ability to supervise the technical and administrative work of other software engineers. 4. The software engineer builds skills and expertise of his/her software engineering discipline to reach standard software engine...

need handson on snowflake , dbt , azure in creating models , pipelines . basic idea on devops

Very Urgent: Data Engineer / Sr.Data Engineer Company: AurumData Solutions Client: MNC Experience: 7+years Location: Jubilee Hills Hyderabad (Customer Location) Job Mode: C2H Duration: 1year + Ext Work Type: Hybrid Mode Note: The selected candidates will be required to work at the customer's location. Key skills: Data Engineer, Python, PySpark, SQL, Databricks, Airflow, Data lake, merging of data, data ingestion, ETL Process, API, FastAPI, AWS (S3, IAM, ECR, Lambda), Rockset, Clickhouse, CrateDB, Avro, Parquet, ORC etc.. Job Summary: A Data Engineer is responsible for overseeing the design, development, and management of data infrastructure and pipelines within an organization. This role inv...

As an experienced candidate with 5-8 years in the field, your role will involve the following key responsibilities: - Integrate with AI models using FAST API/Flask API, including Generative AI models, to solve complex problems. - Experience using frameworks like LangChain, Llamalndex, and agent orchestration tools such as LangGraph, AutoGen, and CrewAI. - Deploy and manage applications in cloud environments (AWS, Azure, or GCP), ensuring scalability, resilience, and cost efficiency. - Experience in integrating with full-stack AI-driven applications and modern front-end frameworks like React, Angular, Vue, or Next.js. - Implement monitoring and observability solutions for real-time tracking, ...

As an ETL Data Engineer at our company, your role will involve designing, developing, and maintaining robust data pipelines using AWS cloud services and Python. You will be responsible for extracting, transforming, and loading large datasets while ensuring data quality, performance, and scalability across our data infrastructure. Key Responsibilities: - Design, develop, and maintain data pipelines using AWS cloud services and Python. - Extract, transform, and load large datasets while ensuring data quality, performance, and scalability. - Work with diverse data sources and formats, including RDBMS and NoSQL databases. - Develop API solutions for data ingestion and integration. - Convert, pro...

We are seeking an experienced and highly skilled Senior Data Engineer to drive data-driven decision-making and innovation. In this role, you will leverage your expertise in advanced analytics, machine learning, and big data technologies to solve complex business challenges. You will be responsible for designing predictive models, building scalable data pipelines, and uncovering actionable insights from structured and unstructured datasets. Collaborating with cross-functional teams, your work will empower strategic decision-making and foster a data-driven culture across the organization. Role & responsibilities 1 Position Overview: We are seeking an experienced and highly skilled Senior Data ...

Join us as a Data Engineer responsible for supporting the successful delivery of Location Strategy projects to plan, budget, agreed quality and governance standards. You'll spearhead the evolution of our digital landscape, driving innovation and excellence. You will harness cutting-edge technology to revolutionise our digital offerings, ensuring unparalleled customer experiences. To be successful as a Data Engineer you should have experience with: Hands on experience in pyspark and strong knowledge on Dataframes, RDD and SparkSQL Hands on Experience in developing, testing and maintaining applications on AWS Cloud. Strong hold on AWS Data Analytics Technology Stack (Glue, S3, Lambda, Lake for...