12368 Big Data Jobs - Page 15

Your future duties and responsibilities: Must have - Strong experience with ETL tools such as Informatica / Talend / DataStage / SSIS (customize as needed) Proficiency in SQL, including complex joins, performance tuning, stored procedures, and functions Solid understanding of data warehousing concepts Experience with PL/SQL / T-SQL or similar Hands-on experience with scheduling/orchestration tools (Control-M, Airflow, Stonebranch, Autosys, etc) Good understanding of relational databases (Oracle, SQL Server, DB2, PostgreSQL) Experience working with large datasets and optimizing data pipelines Famliarity with version control (Git) and CI/CD practices Nice to have Knowledge of Python for automa...

Job Overview Summary The Lead Data Architect is a strategic and technical leadership role responsible for architecting, scaling, and evolving enterprise-grade data platforms that enable advanced analytics, AI/ML, and data-driven decision-making. Reporting to the Senior Director of Data Platforms , this role will lead the design and governance of modern data architectures, drive innovation in AI orchestration, and ensure the delivery of secure, compliant, and high-performing data solutions. This position combines hands-on engineering expertise with architectural vision and cross-functional leadership. The Lead Data Architect will guide engineering teams, influence platform strategy, and estab...

Job Summary: We are looking for an experienced ETL/SQL/Python Testing Engineer to design, develop, and execute test cases for data pipelines, ETL processes, and database systems. The ideal candidate should have strong expertise in SQL querying, ETL validation, Python scripting, and test automation to ensure data accuracy, completeness, and performance.Your future duties and responsibilities - Develop and execute ETL test cases to validate data extraction, transformation, and loading processes. - Has experience in ETL pipeline testing in ADB environment. - Write complex SQL queries to verify data integrity, consistency, and correctness across source and target systems. - Automate ETL testing ...

AWS Redshift, Aurora, AWS Glue, AWS Lambda, etc. Extensive experience with Data gathering and ingestion (from multiple sources), and manipulation, orchestration and optimization on AWS Cloud Spark / Python Developing RESTful APIs Skilled in using ETL tools Proven track record of leading multi-shore teams on sizable Cloud + Data initiatives Working in an Agile Environment Excellent communication, presentation and client interaction skills Ability to work in a very dynamic environment with multiple onshore stakeholders Mandatory Competencies ETL - ETL - AWS Glue Agile - Agile - Extreme Programming Cloud - AWS - AWS Lambda,AWS EventBridge, AWS Fargate Cloud - AWS - Tensorflow on AWS, AWS Glue, ...

Join our data engineering team to build scalable data pipelines and infrastructure We're looking for experienced professionals with expertise in distributed systems, cloud platforms, and big data technologies You'll work with cutting-edge tools to process and analyze massive datasets, collaborate with cross-functional teams, and help drive our data strategy forward Experience with Angular 6+ (preferred) as the product is being built with Angular If you know any other framework such as ReactJS, VueJS open to learn/switch to others when needed Benefits include performance-based bonus and stock options

Role & Responsibilities: This role will have responsibility to take ownership of one or more client engagements at a cross office level encompassing the areas of business intelligence, analytics, Cloud Data management, reporting and mobility; provide thought leadership in developing the firms services and offerings in the Business Technology space and add leverage to the overall leadership span for the aforementioned group in India. The person in this role will be responsible for growing the capabilities in the Business Technology group by developing and managing the existing client relationships and also aid in generating new business engagements. This person will need to fully understand t...

The team has built a Big Data Platform to ingest data from a variety of data sources, standardize metrics and build data & analytics products . Data and ML Platform engineering employs new-age technologies such as Distributed Computing constructs, Real Time model predictions, Deep Learning, Accelerated Compute (GPU); scalable feature stores Cassandra, MySQL, Elastic Search, Solr, Aerospike; scalable programming constructs in, Python and ML Frameworks (TensorFlow, Pytorch, etc). Roles and Responsibilities Drive the data architecture, data modelling, design, and implementation of data applications using standard open source big data tech stack, Data Warehouse / MPP databases and distributed sy...

Role & responsibilities Experience Range in Required Skills: 5 to 8 Years Work Location: Thane, Mumbai Job Description: 1. Minimum 4 years of experience in development of Spark Scala, Strong expertise in Apache Spark, 2. Experience in designing and development of solutions for Big Data using Hadoop ecosystem technologies such as Hadoop Bigdata components like HDFS, Spark, Hive Parquet File format, YARN, MapReduce, Sqoop. 3. Good Experience in writing and optimizing Spark Jobs, Spark SQL etc. Should have worked on both batch and streaming data processing. 4. Experience in writing and optimizing complex Hive and SQL queries to process huge data. good with UDFs, tables, joins, Views etc. 5. Exp...

Project description The primary goal of the project is the modernization, maintenance and development of an eCommerce platform for a big US-based retail company, serving millions of omnichannel customers each week. Solutions are delivered by several Product Teams focused on different domains - Customer, Loyalty, Search and Browse, Data Integration, Cart. Current overriding priorities are new brands onboarding, re-architecture, database migrations, migration of microservices to a unified cloud-native solution without any disruption to business. Responsibilities We are looking for Data Engineer who will be responsible for designing a solution for a big retail company. The main focus is to supp...

Requires a Bachelor''s in Computer Science, Computer Engineering, or related field. 5+ years of experience as a Data Engineer. Experience with Data Warehousing, Operational Data Store, Data technologies and ETL solutions. 4+ years of experience in ETL (IBM Datastage ), SQL, UNIX/Linux scripting, and Big Data distributed systems. 4+ years of experience working with Teradata (Vantage), SQL Server, Greenplum, Hive and delimited text files would be helpful. 3+ years of experience in programming languages like Python, orchestration tools and processes. Experience in building ETL pipelines using Python pandas. Extensive experience in data migration, data analysis, data transformations, conversion,...

What youll do : Collaborate with ZS internal teams and client teams to shape and implement high quality technology solutions that address critical business problems Understand and analyze business problems thoroughly, and translate them into technical designs effectively Design and implement technical features using best practices for the specific technology stack being used Assist in the development phase of implementing technology solutions for client engagements, ensuring effective problem-solving Apply appropriate development methodologies (e.g., agile, waterfall, system integrated testing, mid-development client reviews, embedded QA procedures, unit testing) to ensure successful and tim...

Job description includes : Develops algorithms and applications, performs large scale experimentation, builds data driven apps to translate data into intelligence, and develops solutions for solving business problems using coding Incorporates predictive modeling, statistics and other analysis techniques for collecting, exploring, interpreting and extracting insights from structured and unstructured data Organizes, interprets and structures insights from data, detects data anomalies and makes corrections Performs descriptive statistical analysis, develops visualizations, applications, dashboards, or presentations and communicates insights in a clear and effective way to help the organization ...

Role & responsibilities Experience : 9+ Years. Location : PAN INDIA NP : Immediate to 30 Days. Mandatory-Scala,Spark Scala, and SQL (Strong Exp) (Spark and Hive Mandatory) Should have worked on Strings. Should have Strong experience in Data Sourcing Coding.

Responsibilities Assist in planning, coordinating, and managing engineering projects. Track timelines, deliverables, risks, and resources. Facilitate meetings, sprint planning, and reporting. Support cross-team communication and workflow optimization. Preferred candidate profile Engineering/project coordination roles. Strong understanding of development processes. Excellent communication, organization, and problem-solving skills. Familiarity with project management tools. Perks & Benefits Health Insurance Internet Allowance

Key Responsibilities Design, develop, and maintain scalable Big Data solutions on AWS Build and optimize data ingestion, transformation, and processing pipelines Work with large-scale structured and unstructured datasets Implement ETL/ELT workflows using cloud-native and Big Data tools Optimize performance, cost, and reliability of AWS data services Ensure data quality, security, and governance best practices Mentor junior engineers and contribute to technical best practices Technical Skills (Must Have) AWS Cloud Strong hands-on experience with AWS services such as: S3, EC2, EMR, Glue, Athena, Redshift, Lambda, Airflow IAM, CloudWatch, CloudFormation / Terraform Exposure to Data Lake / Lakeh...

Must To Have Skills: Proficiency in Big Data Analysis Tool and Techniques or Data engineer. - Strong analytical skills to interpret complex data sets and derive actionable insights. - Experience with data processing frameworks such as Apache Hadoop or Apache Spark. - Familiarity with data visualization tools to present findings effectively. - Knowledge of database management systems and data warehousing concepts.

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Apache Spark Good to have skills : NAMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements. You will play a crucial role in developing solutions that align with organizational goals and enhance operational efficiency. Roles & Responsibilities:- Expected to perform independently and become...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As an Software Developer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your Responsibilities may include: * Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, in...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform * Responsibilities: * Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS * Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform * Experience in d...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

Your role and responsibilities As Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Expertise in Data warehousing/ infor...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

Your role and responsibilities As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform...