9747 Big Data Jobs - Page 33

Clear concept of dimensional data modelling(Logical), SCDs, Normalisation and denormalizations. Thoroughly translating the logical models to physical and data flows per the business requirements. Update and optimise the local and metadata models. Could you evaluate the implemented data system for the variances, discrepancies and efficiencies? Troubleshoot and optimise the existing data flows, models and processing jobs by modularising. Explore ways to enhance the data quality and reliability. Strong in writing UDF Previous knowledge of implementing the DQ framework on Spark would be an added advantage. Good in writing programming using Python/Scala/PySpark is a must. Strong knowledge of spar...

6 + years of overall IT experience in Telecom OSS especially in Assurance domain Solution, design, and Implementation - Strong Knowledge of Telecom OSS domain, with excellent experience Service Now for Assuance - Knowledge and experience on Big Data,Data lake solution, KAFKA , Hadoop/Hive. - Experience on Python (pyspark) is essential. - Implementation experience in continuous integration and delivery philosophies and practices specifically on Docker, Git, JenKins - Self driven and highly motivated candidate for a client facing role in a challenging environment

Hands On with advanced SQL, Python etc. Hands On in data profiling Hands On in working on Cloud like Azure and Cloud DW like Databricks Hands On experience on Scheduling tools like Airflow, Control M etc. Knowledgeable on Big Data tools like Spark (python/scala), Hive, Impala, Hue and storage (e.g. HDFS, HBase) Knowledgeable in CICD processes Bitbucket/GitHub, Jenkins, Nexus etc. Knowledgeable managing structured and unstructured data types

Role:Sr Python Developer Must have:Python Spark/Postgres SQL ,Spark / Apache Spark (AWS) Strong development experience using Python and SQL on AWS using Glue, Lambda.

Description: Responsibilities Design and Development: Develop ETL processes using IBM DataStage to extract transform and load data from various data sources to data warehouses or other target systems. Create and maintain data integration solutions to support business requirements. Develop complex DataStage jobs and sequences ensuring efficient performance and scalability. Data Management: Ensure data quality and integrity throughout the ETL process. Implement and monitor data quality checks to ensure accurate data transfer. Perform data analysis and troubleshoot data-related issues as needed. Performance Optimization: Optimize ETL processes for performance and scalability. Monitor and tune D...

Role & responsibilities Design and build a scalable data platform that can ingest, store, manage and stream massive amounts of data while simplifying its analysis and processing to enable rapid development of high-quality data products and services. •Implement and test robust low latency data pipelines for preparing/curating data that will support various data as a service product. •You will work with various cross functional teams to explore data and figure out all the data wrangling needed to clean, curate the data and build the end-to-end data pipeline per requirements. •You will design necessary privacy and security preventive and detective controls into the CICD pipelines of each data p...

Description Experience in following technologiesHadoop Spark Hive Pig SQL query experience in data application development at scale in the Hadoop ecosystem. Bigdata Spark kafka Java/python/scala Hadoop hive design and architecture problem solving skills Roles and responsibilities Drive the technical design of large scale data platforms utilizing modern and open source technologies. Set standards for data engineering functions; design templates for the data management program which are scalable repeatable and simple Serve as a consultant on all aspects of data engineering analytics and data projects Be able to consult on complex data engineering efforts and lead project teams through the solu...

Description The position involves working within our CX Program Implementation team with a focus on building the surveys and reporting. Your day would include working with the business analyst to understand CX requirements shared by the stakeholders and work with the implementation team to develop solutions. The role requires a creative thinker who can solve problems through innovative technology solutions while supporting the development of customer experience programs. Responsibilities ? Configure the Medallia surveys and reporting setup as per the requirements shared by the Business Analyst. ? Build Auto Importers for the new survey programs. ? Conduct file inflow reviews to understand th...

Experienced data modelers, SQL, ETL, with some development background to provide defining new data schemas, data ingestion for Adobe Experience Platform customers. Interface directly with enterprise customers and collaborate with internal teams. 10+ years of strong experience with data transformation & ETL on large data sets/5+ years of Data Modelling experience (i.e., Relational, Dimensional, Columnar, Big Data) 5+ years of complex SQL or NoSQL experience Experience in advanced Data Warehouse concepts

Description 1.Hands on industry experience in design and coding from scratch in AWS Glue-Pyspark with services like S3 DynamoDB StepFunctions etc. 2.Hands on industry experience in design and coding from scratch in Snowflake 3.Experience in Pyspark/Snowflake 1 to 3 years with overall around 5 years of experience in building data/analytics solutions Level Senior Consultant or below Named Job Posting? (if Yes - needs to be approved by SCSC) Additional Details Global Grade C Level To Be Defined Named Job Posting? (if Yes - needs to be approved by SCSC) No Remote work possibility Yes Global Role Family 60236 (P) Software Engineering Local Role Name 6361 Software Engineer Local Skills 59383 AWS G...

Description Primary/Mandatory Experience - Experience building data pipeline in Python along with AWS services (S3 SNS CloudWatch Lambda Step Function etc) - Proficient in AWS Server less technologies - Technical knowledge of Extract/Transform/Load (ETL) solutions for running analytic projects on the cloud - Candidate must have hands-on technical experience on the AWS cloud native technologies along with traditional ETL tools -Snowflake & DWH experience is a plus Daily Activity Excellent written and verbal communication skills and be able to lead meetings with technical peers and clients regarding the solution designs. Ability to communicate with the Business Analysts Data Modellers Cloud Ar...

Description Primary/Mandatory Experience - Experience building data pipeline in Python along with AWS services (S3 SNS CloudWatch Lambda Step Function etc) - Proficient in AWS Server less technologies - Technical knowledge of Extract/Transform/Load (ETL) solutions for running analytic projects on the cloud - Candidate must have hands-on technical experience on the AWS cloud native technologies along with traditional ETL tools -Snowflake & DWH experience is a plus Daily Activity Excellent written and verbal communication skills and be able to lead meetings with technical peers and clients regarding the solution designs. Ability to communicate with the Business Analysts Data Modellers Cloud Ar...

JD -7+ years of hands on experience in Python especially dealing with Pandas and Numpy Good hands-on experience in Spark PySpark and Spark SQL Hands on experience in Databricks Unity Catalog Delta Lake Lake house Platform Medallion Architecture Azure Data Factory ADLS Experience in dealing with Parquet and JSON file format Knowledge in Snowflake.

What youll do : Collaborate with ZS internal teams and client teams to shape and implement high quality technology solutions that address critical business problems Develop a deep understanding of the business problems and effectively translate them into technical designs Lead modules and workstreams within projects while participating in hands on implementation Work with technical architects to validate the technical design and implementation approach Apply appropriate development methodologies and best practices to ensure exceptional client and project team experience Support the project lead in project delivery, including project planning, people management, staffing, and risk mitigation ...

We are seeking a talented individual to join our Tech team at MMC Corporate.This role will be based in Gurgaon/ Noida/Pune/Mumbai. This is a hybrid role that has a requirement of working at least three days a week in the office. Senior Principal Engineer Data Engineering What can you expect? As a Data Engineer, you will be responsible for designing and implementing scalable data pipelines and AI Based solution using Databricks. You will handle end-to-end ETL/ELT processes, manage large datasets, and work with tools like Python, PySpark, and AWS S3 to ensure data is transformed and optimized for analytical use. Youll work on cutting-edge cloud and hybrid data projects, transforming raw data i...

Demonstrated considerable experience (minimum 5-7 Years) implementing Data platform/Big data projects on Hyperscalars platform. (Primarily on AWS/Databricks & Google cloud) Extensive working experience on implementing data warehouse, data lake, data mart projects (5-8 years) Strong solution/architect skills in building Data engineering pipelines. (> 5 Years) Hands on experience working in AWS (glue, redshift, s3, athena)/Databricks/Spark (pyspark preferred) /Snowflake/Google cloud (GCS, Big query, dataproc, dataflow, composor). Experience working on Real team data engineering projects Having Data engineering Certifications will be added advantage Good to have Knowledge of AI/ML skills, ML En...

About the Team At Nutanix, you would be joining the Next Gen- Data Engineering team comprised of more than 40 skilled professionals. Our team thrives in an open culture with a flat hierarchy, where collaboration and innovation are at the heart of everything we do. You can expect to work alongside talented individuals who are passionate about pioneering new solutions in the tech industry. You will report to the Senior Engineering Manager, who is dedicated to providing guidance and support for your career growth. The work setup at Nutanix is a hybrid model, with employees expected to be in the office 2 days a week. This structure allows for both in-person collaboration and flexibility for remo...

The Senior Professional, Data Engineering job designs, builds and maintains complex data systems that enable data analysis and reporting. With minimal supervision, this job ensures that large. Required Candidate profile DATA FORMATS: Examines and resolves appropriate data formats to improve data usability and accessibility across the organization.

Demonstrated considerable experience (minimum 5-7 Years) implementing Data platform/Big data projects on Hyperscalars platform. (Primarily on AWS/Databricks & Google cloud) Extensive working experience on implementing data warehouse, data lake, data mart projects (5-8 years) Strong solution/architect skills in building Data engineering pipelines. (> 5 Years) Hands on experience working in AWS (glue, redshift, s3, athena)/Databricks/Spark (pyspark preferred) /Snowflake/Google cloud (GCS, Bigquery, dataproc, dataflow, composor) Experience working on Real team data engineering projects Having Data engineering Certifications will be added advantage Good to have Knowledge of AI/ML skills, ML Engi...

Senior Engineer - Data Engineering, with Automotive domain having 3 - 7 years of experience, is expected to lead the design, development, and optimization of data engineering solutions leveraging big data technologies such as Hadoop and Apache Spark. The ideal candidate will possess a robust understanding of distributed data processing and demonstrate expertise in Python for data manipulation and analysis using libraries like Pandas and NumPy. Familiarity with cloud services, particularly Google Cloud Platform (GCP), is critical for deploying and managing applications in a cloud environment. Proficiency in Terraform for infrastructure as code is necessary, along with knowledge of Apache Airf...

Experienced Data Engineer with 5+ years of hands-on development using Python (preferred), along with strong proficiency in Scala, Java, or C#. Skilled in designing and building high-performance, scalable data pipelines and architectures. Specialized in working with PySpark on Databricks, and experienced across a range of data engineering tools including Snowflake, BigQuery, Apache Spark, Hive, Hadoop, Cloudera, and Redshift. Key Competencies: Programming & Development: 5+ years of experience developing data-centric applications using Python (preferred), with additional expertise in Scala, Java, or C#. Big Data & Cloud Platforms: Deep experience with cloud-based and on-premise data platforms ...

Role Proficiency: Leverage expertise in a technology area (e.g. Big Data Cluster) and responsible for Architecture for small to mid size projects. Outcomes: Design and develop the framework; be the technical owner for a project in one technology area Guide and review technical delivery by internal teams Resolve design issues and deliver/own design for a complex module for projects of the following nature: architecture enhancements small sized development and maintenance projects Review design and code ensuring adoption of industry best practices Measures of Outcomes: Number of reusable components / processes developed Number of times components / processes reused Delivery efficiency (Adheren...

Role: Senior Data Engineer (Palantir Foundry) Experience: 5 to 12 Years Required Skills & Experience: Bachelor's or Master's degree in Computer Science, Engineering, Mathematics, or related field. Minimum 6 years of experience in system engineering or software development. At least 4 years of hands-on experience in building robust ETL pipelines for batch and streaming data ingestion . Proficiency in Python and PySpark for distributed computing; knowledge of SQL and Scala is a plu s. Strong experience with Palantir Foundry platform components (Code Repository, Ontology Manager, Workshop App, APIs, SDK). Solid understanding of Big Data technologies , Hadoop ecosystem, Spark, and distributed co...

We are looking for a dynamic professional who brings a unique blend of software development expertise and business analysis acumen. The ideal candidate will have hands-on experience in coding and system design, coupled with a strong understanding of business processes and stakeholder management. Key Responsibilities: Collaborate with stakeholders to gather and analyze business requirements. Translate business needs into technical specifications and solutions. Participate in system design, development, and testing phases. Work closely with development teams to ensure business requirements are met. Create and maintain documentation including BRDs, FRDs, and user stories. Conduct gap analysis a...

As a Java Developer in NYC, NY, your role will involve providing comprehensive technical expertise across multiple technology areas. You will be responsible for solving high-impact, complex business and system issues, developing methodologies and standards for application analysis and testing, managing resources, mentoring junior analysts, and ensuring high-quality deliverables. Additionally, you will liaise with QA and support teams to ensure post-release success while assessing risk and maintaining compliance with Citis policies and standards. Key Responsibilities: - Provide comprehensive technical expertise across multiple technology areas - Solve high-impact, complex business and system ...