9690 Big Data Jobs - Page 4

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: * Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. * Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. * Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. Required education ...

As an Software Developer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: * Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, in...

As an Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorpora...

As Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: * Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporat...

As Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: * Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporat...

As an Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing.Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets.In this role, your responsibilities may include:Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporatin...

Key Responsibilities To build scalable, schema-driven, and compliant data pipelines powering real-time analytics, search, and long-term storage across Kafka, ClickHouse, PostgreSQL, and OpenSearch ecosystems. ClickHouse Optimization & Scaling — Research, plan, and implement ClickHouse cluster optimizations for multi-tenant workloads. Design rollup and retention strategies to balance performance, cost, and compliance. Big Data Tooling & Integrations — Introduce and evaluate Big Data components such as S3/object storage, serverless databases, Kafka, Kafka Connect, and Kafka Schema Registry, adopting modern data-warehouse practices. Design data-in-motion and data-at-rest pipelines with strong e...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs.\Your primary responsibilities include:Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements.Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization.Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelor's...

Role Description : As a Technical Lead - Cloud Data Platform (Azure) at Incedo, you will be responsible for managing and optimizing the Microsoft Azure environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or GCP and have experience with data warehousing technologies such as Azure Synapse Analytics. You will be responsible for configuring and optimizing the Azure environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the...

Role Description : As a Senior Cloud Data Platform (Azure) Engineer at Incedo, you will be responsible for managing and optimizing the Microsoft Azure environment, ensuring its performance, scalability, and security. You will work closely with data analysts and data scientists to develop data pipelines and run data science experiments. You will be skilled in cloud computing platforms such as AWS or GCP and have experience with data warehousing technologies such as Azure Synapse Analytics. You will be responsible for configuring and optimizing the Azure environment, ensuring that data pipelines are efficient and accurate, and troubleshooting any issues that arise. You will also work with the ...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs. Your primary responsibilities include: * Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. * Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. * Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too. Required education ...

Dear Candidates, We have urgent openings for Big data engineer - CMM L5 Client- Permanent position Location-Chennai/Bangalore-Hybrid mode Exp-4-15yrs NP- Immediate to 20days max(Serving NP only) Job Description: Primary Skills: Hadoop, Hive, Python, Spark, SQL, Airflow... Through and hands on knowledge on Python and Spark Programming and writing efficient, reusable, and reliable Python code. Very good proficiency in Hadoop, Python, Airflow, SQL and Spark programming. Implementation experience of Spark Core, Spark SQL and Spark Streaming Working with Spark in combination Hadoop Ecosystem. if anyone interested pls share your resume to Manasa@skyonn.com

Job Summary We are seeking a highly skilled Sr. Developer with 6 to 10 years of experience to join our team in a hybrid work model. The ideal candidate will have expertise in Databricks SQL Databricks Delta Lake Databricks Workflows and PySpark. This role involves developing and optimizing data solutions to drive business insights and efficiencies. The position is a day shift role with no travel requirements. Responsibilities Develop and implement data solutions using Databricks SQL to enhance business intelligence capabilities. Optimize data processing workflows with Databricks Delta Lake to ensure data integrity and performance. Design and manage Databricks Workflows to automate data pipel...

Key responsibilities Design, develop, and maintain scalable Scala applications and systems. Perform software analysis and write code according to application specifications. Collaborate with developers, testers, and product managers to ensure applications meet standards. Troubleshoot and debug complex issues in Scala applications. Contribute to architectural decisions and recommend improvements to processes and infrastructure. Participate in the entire software development lifecycle, including testing and deployment. Implement data validation and quality checks, especially in data-intensive applications. Contribute to CI/CD pipelines and automate build, test, and deployment processes.

We are looking for a highly skilled and motivated Business Analyst to join our team in Hyderabad. The ideal candidate will have 1-2 years of experience in business analysis, with excellent communication and problem-solving skills. Roles and Responsibility Define data requirements and collect relevant data for analysis. Process, clean, and analyze data to identify trends and patterns. Develop and maintain databases and data systems for business intelligence. Collaborate with cross-functional teams to improve decision-making processes. Identify areas for efficiency and automation in data analysis processes. Provide inputs for complex Data Science and Big Data Mining projects. Job Requirements ...

SUMMARY Job Description: As a Lead SRE, you will architect, implement, and manage cloud and data reliability strategies across large-scale applications and client environments. This role emphasizes system scalability, resilience, automation, and mentoring junior engineers. Responsibilities: Lead design and implementation of reliability frameworks for applications and data pipelines. Architect and secure scalable cloud infrastructure (AWS / Azure). Drive incident management, service availability, and disaster recovery planning. Automate provisioning and configuration using Terraform, Ansible, or CloudFormation. Oversee monitoring systems, SLO/SLI tracking, and cost optimization. Mentor junior...

Role & responsibilities Utilize strong SQL and Python skills to extract, transform, and analyze large datasets, uncovering insights and trends. Understanding of role of statistical models and analytics in business decision-making Apply statistical methodologies to interpret data, conduct hypothesis testing, and provide actionable recommendations for business improvements. Work closely with cross-functional teams to understand business requirements and translate them into data-driven solutions. Create clear and insightful visualizations using tools like Qliksense/Looker to effectively communicate findings to both technical and non-technical stakeholders. Create graphical visualizations and KP...

Key Update: PySpark skill is no longer required for this role. Revised JD: Must Have: SQL: Ability to read, write, test, and maintain complex SQL queries BigQuery: Working knowledge, including aspects like internal vs external tables, partitioning, clustering, etc. Python: Hands-on expertise in writing Python application code Big Data Engineering: Experience in Big Data Engineering projects Good to Have (Added Advantage): Knowledge/experience with DBT and Alembic (availability of these skills will be a big push) Experience Requirement: Mid-level engineer with 7-8 years of experience matching the above JD

4+ Years Data Engineering Experience, 5+ years of consulting experience Completed Data Engineering Professional certification & required classes Minimum 2-3 projects delivered with hands-on experience in Databricks Completed Apache Spark Programming with Databricks, Data Engineering with Databricks, Optimizing Apache Spark on Databricks Experience in Spark and/or Hadoop, Flink, Presto, other popular big data engines Familiarity with Databricks multi-hop pipeline architecture

About Bazaarvoice At Bazaarvoice, we create smart shopping experiences. Through our expansive global network, product-passionate community & enterprise technology, we connect thousands of brands and retailers with billions of consumers. Our solutions enable brands to connect with consumers and collect valuable user-generated content, at an unprecedented scale. This content achieves global reach by leveraging our extensive and ever-expanding retail, social & search syndication network. And we make it easy for brands & retailers to gain valuable business insights from real-time consumer feedback with intuitive tools and dashboards. The result is smarter shopping: loyal customers, increased sal...

Role: Associate Architect - Data Experience Level: 6 to 9 Years Work location: Mumbai, Bangalore, Trivandrum Notice Period: 0-45 days only Role & Responsibilities: 5+ years of experience in building and managing Data Lakes, Data Warehouse, Data Integration, Data Migration and Business Intelligence/Artificial Intelligence solutions on Cloud (GCP/AWS/Azure) Ability to understand business requirements, translate them into functional and non-functional areas, and define non-functional boundaries in terms of Availability, Scalability, Performance, Security, Resilience etc. Experience in architecting, designing, and implementing end-to-end data pipelines and data integration solutions for varied s...

5-10 years of experience in database development or a related field. Proven experience with database design, development, and management. Experience working with large-scale databases and complex data environments Experience with data modelling and database design. Knowledge of database performance tuning and optimization. Architect, Develop and maintain tables, views, procedures, functions and packages in Database MUST HAVE Performing complex relational databases queries using SQL (AWS RDS for PostgreSQL) and Oracle PLSQL MUST HAVE Familiarity with ETL processes and tools (AWS Batch, AWS Glue etc) MUST HAVE Familiarity with CI/CD Pipelines, Jenkins Deployment, Git Repository MUST HAVE Perfo...

What you will do In this vital role you will responsible for designing, developing, and maintaining software solutions for Research scientists. Additionally, it involves automating operations, monitoring system health, and responding to incidents to minimize downtime. You will join a multi-functional team of scientists and software professionals that enables technology and data capabilities to evaluate drug candidates and assess their abilities to affect the biology of drug targets. This team implements scientific software platforms that enable the capture, analysis, storage, and reporting for our Large Molecule Discovery Research team (Design, Make, Test and Analyze processes). The team als...

What you will do Role Description: The role is responsible for the design, integration, and management of high performance computing (HPC) systems that encompass both hardware and software components into the organizations network infrastructure. This individual will be responsible for all activities related to handling and supporting the Business and platforms including system administration, as well as incorporating new technologies under the challenge of a sophisticated and constantly evolving technology landscape. This role involves ensuring that all parts of a system work together seamlessly to meet the organizations requirements. Roles & Responsibilities : Implement, and manage cloud-b...

What you do We are seeking a seasoned Sr Manager of Data Engineering to drive the development and implementation of our data strategy for Manufacturing and Clinical Supply Chain. The ideal candidate possesses a strong blend of technical expertise and data-driven problem-solving skills. As a Sr. Manager Data Engineering, you will play a crucial role in leading, designing, building, and optimizing our data pipelines and platforms while mentoring junior engineers. Roles & Responsibilities: Contribute to the design, development, and implementation of data pipelines, ETL/ELT processes, and data integration solutions. Take ownership of data pipeline projects from inception to deployment, managing ...