6547 Big Data Jobs - Page 27

Diverse Lynx is looking for Data Engineer to join our dynamic team and embark on a rewarding career journey Liaising with coworkers and clients to elucidate the requirements for each task. Conceptualizing and generating infrastructure that allows big data to be accessed and analyzed. Reformulating existing frameworks to optimize their functioning. Testing such structures to ensure that they are fit for use. Preparing raw data for manipulation by data scientists. Detecting and correcting errors in your work. Ensuring that your work remains backed up and readily accessible to relevant coworkers. Remaining up-to-date with industry standards and technological advancements that will improve the q...

Diverse Lynx is looking for Big Data Developer to join our dynamic team and embark on a rewarding career journey Design, develop, and maintain big data solutions to meet business requirements and support data-driven decision making Work with stakeholders to understand their data needs and determine how to best use big data technologies to meet those needs Design and implement scalable, high-performance big data architectures, using technologies such as Hadoop, Spark, and NoSQL databases Extract, transform, and load large data sets into a big data platform for analysis and reporting Write complex SQL queries and develop custom scripts to process big data Collaborate with data scientists, data...

Diverse Lynx is looking for Data Engineer to join our dynamic team and embark on a rewarding career journey Liaising with coworkers and clients to elucidate the requirements for each task. Conceptualizing and generating infrastructure that allows big data to be accessed and analyzed. Reformulating existing frameworks to optimize their functioning. Testing such structures to ensure that they are fit for use. Preparing raw data for manipulation by data scientists. Detecting and correcting errors in your work. Ensuring that your work remains backed up and readily accessible to relevant coworkers. Remaining up-to-date with industry standards and technological advancements that will improve the q...

Develop metadata pipelines to tag documents with freshness, ownership, and other context for better filtering. Implement caching and multi-region replication to reduce query latency. Monitor data retrieval accuracy and log source citations to improve AI trustworthiness. Automate ingestion and embedding generation for unstructured data into vector databases like Zilliz, Pinecone, or OpenSearch. 1+ years of data engineering experience Experience with data modeling, warehousing and building ETL pipelines Experience with one or more query language (e.g., SQL, PL/SQL, DDL, MDX, HiveQL, SparkSQL, Scala) Experience with one or more scripting language (e.g., Python, KornShell) Experience with big da...

We are seeking a Data Engineer I who will support a GenAI-powered insights assistant initiative by building and scaling ingestion and embedding pipelines for unstructured WW FBA knowledge bases. Your role ensures the retrieval-augmented generation system accesses fresh, relevant document embeddings to enhance AI-driven insights and user query satisfaction. Build batch and streaming data pipelines using Spark and AWS streaming services. Implement automated checks to ensure data consistency across different data types. Define and maintain data contracts with source teams to keep schemas consistent. Develop cross-domain metadata services linking structured and unstructured data catalogs. Create...

Lead your team to build a brand new product with quality, robustness and Amazon sized scale as top priorities. Contribute to architecture and design discussions for full stack software. Lead your team to evolve our service architecture with quality, robustness and Amazon sized scale as top priorities. You will regularly take part in deep-dive troubleshooting exercises and drive technical post-mortem discussions to identify the root cause of complex issues. Set a vision for your team and create product roadmaps. Help your team sort out technical and product requirements and priorities. Deliver product roadmap items and cross-team projects. Delight your customers. Internalize our customer chal...

Key Responsibilities: Develop and implement traditional machine learning algorithms. Deploy at least one model in a production environment. Write and maintain Python code for data science and machine learning projects.Preferred Qualifications: Knowledge of Deep Learning (DL) techniques. Experience working with Generative AI (GenAI) and Large Language Models (LLM). Exposure to Langchain.

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelo...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the client’s needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our client’s business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bache...

Advanced Programming Skills in Python,Scala,GoStrong expertise in developing and maintaining microservices in Go (or other similar languages), with the ability to lead and mentor others in this area. Extensive exposure in developing Big Data Applications ,Data Engineering ,ETL and Data Analytics . Cloud ExpertiseIn-depth knowledge of IBM Cloud or similar cloud platforms, with a proven track record of deploying and managing cloud-native applications. Leadership and CollaborationAbility to lead cross-functional teams, work closely with product owners, and drive platform enhancements while mentoring junior team members. Security and ComplianceStrong understanding of security best practices and ...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. Required education Bachel...

As an Software Developer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, inco...

We are currently seeking a Software Qlty Assurance Sr. Specialist to join our team in Chennai, Tamil Ndu (IN-TN), India (IN). ETL, PL/SQL, Databricks and PySpark experience, with experience migrating to the cloud

We are currently seeking a AI Data Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN). Job Title: Data Science & AIML, GenAI Lead/Engineer Key Responsibilities: Develop and implement traditional machine learning algorithms. Deploy at least one model in a production environment. Write and maintain Python code for data science and machine learning projects. Preferred Qualifications: Knowledge of Deep Learning (DL) techniques. Experience working with Generative AI (GenAI) and Large Language Models (LLM). Exposure to Langchain. #GenAI

We are currently seeking a Snowflake Engineer - Digital Solution Consultant Sr. Analyst to join our team in Hyderabad, Telangana (IN-TG), India (IN). Experience with other cloud data warehousing solutions. Knowledge of big data technologies (e.g., Apache Spark, Hadoop). Experience with CI/CD pipelines and DevOps practices. Familiarity with data visualization tools.

Analyze and report on data quality statistics for P&C policy data. Data includes limits, deductibles, exposures, and other key elements of a policy. Analyst will work with Operational Leaders to consult on needed data Required Candidate profile Ability to construct visual representations of large data sets Proficient in big data analysis tools, including SQL, R, or Python. Experience with visualization tools, such as PowerBI.

Senior data engineers at Thoughtworks are engineers who build, maintain and test the software architecture and infrastructure for managing data applications. They are involved in developing core capabilities which include technical and functional data platforms. They are the anchor for functional streams of work and are accountable for timely delivery. They work on the latest big data tools, frameworks and offerings (data mesh, etc.), while also being involved in enabling credible and collaborative problem solving to execute on a strategy. Job responsibilities You will develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solution...

MANDATORY REQUIREMENTS Database-MS SQL Cloud Technologies-Microsoft Azure Servers-Windows, Linux High Availability-Fail-over Cluster Instance, AAG, Replication Documentation-Technical Documentations (Set-up, Technical Tutorials, Flowcharts, Architecture Diagrams, UMLs, etc Others-Scripting (T-SQL) NICE TO HAVE Database Postgre, mySQL Cloud Services AWS Development & Reporting Tools Data Tools, SSRS, SSIS Monitoring Tools SolarWinds DPA, Azure Monitor, AWS Monitor Others Hadoop Big Data Technologies. Capabilities Critical thinker and problem solving skills Good Communication skills Sense of ownership and pride in your performance and its impact on companys success Configure, maintain high ava...

Requirements 6+ years of experience Proficiency in Java, Spring Boot, object databases, ElasticSearch/Solr, Practice in using AWS cloud, Docker & Kubernetes, REST APIs Experience in building scalable and high-performance systems Strong communication skills in English (B2+) Nice-to-have: Knowledge of Python, ETL experience, and big data solutions Responsibilities Maintenance of a large modern search platform Handling production issues and incidents Optimizing and maintaining existing code for performance and availability Ensuring high performance and availability of the system Engage in the Release Process Team Information Work within a SAFe, scrum kanban methodology, and agile approach Colla...

Overview : We are looking for a hands on QE VP lead capable of managing a set of technofunctional quality engineers to be able to drive critical multi-year initiatives on the TaxSmart platform. The individual should be able to work with the team and the stakeholders to build a plan for these initiatives, think through the various flavors of testing that might be needed to ensure platform delivery, quality and stability. Additionally, they should be able to drive multiyear quality initiatives around enhancing the QE team s capabilities to achieve specific KRAs around F2B testing, defect metrics, insprint automation, reusability in frameworks, prod issue leakage, fungibility and cross-skilling...

Responsibilities: Lead and mentor a high-performing teams of local and remote engineers Prioritize team workload, allocate tasks effectively, and ensure team members have the resources to succeed Provide technical expertise and guidance to the team Evaluate and mentor adherence to coding standards, best practices, and architectural guidelines Oversee the design, development, maintenance, scalability, reliability, and performance of the Industrial Systems data platform pipelines and architecture. Contribute to the long-term strategic direction of the Data Platform with a focus on enterprise use Enforce and ensure data quality, data governance, and security standards Identify and consolidate c...

Data Bricks and Python Engineer Job Responsibilities 1. Software Development: - Design, develop, test, and deploy high-performance and scalable data solutions using Python,PySpark, SQL - Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications. - Implement efficient and maintainable code using best practices and coding standards. 2. Databricks Platform: - Work with Databricks platform for big data processing and analytics. - Develop and maintain ETL processes using Databricks notebooks. - Implement and optimize data pipelines for data transformation and integration. 3. Continuous Learning: - Stay updated on the latest indust...

The Oracle Cloud Infrastructure (OCI) Security and Compliance Platform Engineering organization presents a rare opportunity to contribute to the development of next-generation, AI-driven cybersecurity solutions at cloud scale. This effort centers on ingesting and processing massive volumes of telemetry and security event data across OCI, leveraging advanced techniques including generative AI (GenAI), large language models (LLMs), and machine learning (ML) to build intelligent detection, response, and mitigation systems. The goal is to deliver autonomous, adaptive security capabilities that protect OCI, Oracle, and our global customer base against evolving threat landscapes. Inviting you to b...

As a member of the Support organization, your focus is to deliver post-sales support and solutions to the Oracle customer base while serving as an advocate for customer needs. This involves resolving post-sales non-technical customer inquiries via phone and electronic means, as well as, technical questions regarding the use of and troubleshooting for our Electronic Support Services. A primary point of contact for customers, you are responsible for facilitating customer relationships with Support and providing advice and assistance to internal Oracle employees on diverse customer situations and escalated issues. Career Level - IC3 Responsibilities As a Sr. Support Engineer, you will be the te...

Job Summary We are seeking a talented Data Engineer to join our dynamic team, focusing on big data solutions for the banking industry. In this role, you will lead the design and development of Big Data Warehouse and Data Lake systems, playing a crucial part in our clients' digital transformation journeys. Your expertise in data engineering will empower financial institutions to harness the power of their data, enabling better decision-making and enhanced customer experiences. Key Responsibilities Big Data Warehouse and Data Lake Development : Lead the design and implementation of Big Data Warehouse and Data Lake systems, utilizing the Hadoop ecosystem and cloud technologies. Data Pipeline Ar...