34935 Spark Jobs

Job Summary: We are seeking an experienced and passionate Senior Data Engineer to join our dynamic data engineering team. The ideal candidate will have deep expertise in building scalable data pipelines and distributed data processing systems using Python , Apache Spark , Kafka , and Snowflake/Databricks . This role involves designing, developing, and optimizing high-performance data solutions that enable advanced analytics and business intelligence across the organization. Experience: 7 12 years Location : Bangalore Mode of Work : Full time hybrid Key Responsibilities: Design, build, and maintain robust, scalable data pipelines using Python and Apache Spark. Integrate and process large volu...

Dear Candidate Greetings from TATA Consultancy Services Job Openings at TCS Skill Data Engineer (Spark scala) Exp range: 5yrs to 10 yrs Location: Chennai/Pune/Bangalore/Hyd/Delhi Notice period 30 days Pls find the Job Description below. 1)should be able to design strategies and programs to collect, store, analyse and visualize data from various sources. 2) Should be able to develop big data solution recommendations and ensure implementation of the chosen big data solution. 3) Needs to be able to program, preferably in different programming/scripting languages such as Scala, Python, Java, Pig or SQL. 4) Proficient knowledge in Big data frameworks Spark, Map Reduce, 5) Should have an understan...

Senior Solution Architect: RoleOverview Experienced Senior Solution Architect Data to lead the design and governance ofenterprise-wide data platforms and analytics solutions. Lead design andoptimisation of the data ecosystem for a large engineering enterprise with adiverse application landscape that includes multiple ERPs (Baan, Oracle, SAP), cloud platforms, and SaaS solutions. Demonstratedtechnical expertise with strong architectural leadership is a must for therole. Key Responsibilities Define and govern the enterprise data architecture, ensuring alignment with business strategy and IT standards. Design end-to-end data solutions across ingestion, storage, modelling, analytics, and governa...

Developing scalable data pipelines and streaming applications. Working with real-time data streams (Kafka). Processing large datasets using Spark. Building microservices or backend services in Java with Spring Boot. Ensuring data processing is efficient and reliable.

Data Engineer (5-7 years exp) Engineer with strong expertise in PySpark/Spark, along with solid knowledge of data warehouse fundamentals and Databricks. The role involves building and optimizing scalable data pipelines, implementing data models, and managing large datasets across cloud environments. Python is good to have, while PySpark/Spark is a must. Key Responsibilities: Design, build, and maintain scalable ETL/ELT pipelines on Databricks platform. Apply strong data warehouse fundamentals in schema design, dimensional modeling, and optimization. Develop and optimize distributed data processing workflows using PySpark/Spark. Collaborate with analysts, data scientists, and business teams t...

Skills Qualifications: Excellent communication and interpersonal skills. Strong writing skills and a good sense of tone for online engagement. Proficiency in MS Excel or Google Sheets for database management and reporting. Basic understanding of CRM or community management tools (preferred). Active on social media and aware of current digital trends. Organized, proactive, and able to multitask. Prior experience in community management, social media, or content creation (preferred but not mandatory). Duties: Key Responsibilities: Engage with users across social media platforms, forums, and community groups. Respond to queries, comments, and feedback in a timely and empathetic manner. Support ...

The Principal Data Engineer will be a senior individual contributor responsible for designing, building, and optimizing advanced data solutions that power enterprise-wide analytics, AI/ML, and data-driven decision-making. Reporting directly to the Senior Director of Data Platforms, this role will serve as a technical expert and thought partner in scaling modern data architectures, enabling AI orchestration, and delivering robust, secure, and compliant data products. This position is highly hands-on, requiring expertise in data engineering, Databricks, Python, AI/ML platforms, and orchestration frameworks. The Principal Engineer will work across functional teams to design and implement high-p...

Develop, test, and maintain ETL pipelines from TMS source systems to canonical layers. Implement data transformation, cleansing, and validation logic. Support QA by fixing data defects and ensuring completeness of load. Mandatory Competencies ETL - ETL - Tester Beh - Communication ETL - ETL - AWS Glue Data Science and Machine Learning - Data Science and Machine Learning - Databricks

Basic Qualifications Bachelors degree in Computer Science, AI/ML, or a related quantitative field. 3-5 years of experience in AI/ML engineering, with hands-on exposure to GenAI, LLMs, NLP, and agentic architectures. Proven experience deploying enterprise-grade AI agents, including pre-launch testing, HITL workflows, and business UAT sign-off. Familiarity with multi-modal AI (text, image, audio) and unstructured data processing using Databricks Genie, AI Foundry, and AI Search. Experience working in agile, cross-functional teams and delivering measurable impact through automation and intelligent systems. Duties Responsibilities Translate business requirements into scalable and well-documented...

Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Do Oversee and support process by reviewing daily transactions on performance parameters Review performance dashboard and the scores for the team Support the team in improving performance parameters by providing technical support and process guidance Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions Ensure standard processes and procedures are followed to resolve all client queries Resolve cl...

Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do 1. Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information needs, studying systems flow, data usage and work processes Investigating problem areas followed by the software development life cycle Facilitate root cause analysis of the system issues and problem statement Identify ideas to improve system performance and impact availability Analyze client requirements and convert requirements to fe...

Job Title: Sr. Manager Data Warehousing & Business Intelligence Location: Pune Location: Pune Shift Time: EST/PST About the Role: We are seeking a highly skilled and experienced Sr. Manager – Data Operations to take end-to-end ownership of our data management processes. You will be responsible for managing and optimizing our existing data assets sourced internally and externally, ensuring data is reliable, accessible, and actionable. You will work closely with the architecture and analytics teams to build scalable and efficient data operations that power our business insights and decision-making. Key Responsibilities: End-to-End Data Ownership: Manage and oversee the complete lifecycle of da...

Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Do Oversee and support process by reviewing daily transactions on performance parameters Review performance dashboard and the scores for the team Support the team in improving performance parameters by providing technical support and process guidance Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions Ensure standard processes and procedures are followed to resolve all client queries Resolve cl...

Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do 1. Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information needs, studying systems flow, data usage and work processes Investigating problem areas followed by the software development life cycle Facilitate root cause analysis of the system issues and problem statement Identify ideas to improve system performance and impact availability Analyze client requirements and convert requirements to fe...

Key Responsibilities: Operating and calibrating ICP- OES instrument.Performing analysis of Fine purity samples (in-process and finished products).Maintaining test records, reports, and related documentation.Ensuring compliance with regulatory guidelines as per IS 16900 / IS 16901 or its equivalent ISO 15093 and ISO 15096.Investigating out-of-specification (OOS) results and non-conformance issues. Technical Skills: A strong understanding of analytical chemistry, common advancements, and challenges in the field is crucial. Method development and verification skills as per the testing requirement. Education & Experience: Bachelor's or Master's degree in Chemistry, Analytical Chemistry, Chemical...

About The Role Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : NA Minimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform components. Your typical day will involve collaborating wit...

About The Role Project Role : Data Platform Engineer Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Platform Engineer, you will assist with the data platform blueprint and design, encompassing the relevant data platform components. Your typical day will involve collaborating wit...

About The Role Project Role : Data Architect Project Role Description : Define the data requirements and structure for the application. Model and design the application data structure, storage and integration. Must have skills : AWS Architecture Good to have skills : NA Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Engineering Lead, you will spearhead the design and development of cutting-edge technology solutions across diverse PODs, ensuring alignment with architectural standards and business objectives. You will leverage extensive experience and leadership skills to orchestrate collaborative efforts among cross-functio...

About The Role Project Role : Data Architect Project Role Description : Define the data requirements and structure for the application. Model and design the application data structure, storage and integration. Must have skills : AWS Architecture Good to have skills : NA Minimum 15 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Architect, you will define the data requirements and structure for the application. Your typical day will involve modeling and designing the application data structure, storage, and integration, ensuring that the architecture aligns with business needs and technical specifications. You will collaborate with...

About The Role Project Role : Data Architect Project Role Description : Define the data requirements and structure for the application. Model and design the application data structure, storage and integration. Must have skills : AWS Architecture Good to have skills : NA Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Architect, you will define the data requirements and structure for the application. Your typical day will involve modeling and designing the application data structure, storage, and integration, ensuring that the architecture aligns with business needs and technical specifications. You will collaborate with...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : O9 Solutions Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :- As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with various teams to understand their needs, developing innovative solutions, and ensuring that applications are aligned with business objectives. You will engage in problem-solving a...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Microsoft Azure Databricks, Microsoft Azure Data Services, Apache Spark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the applications are aligned ...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Data Building Tool Good to have skills : Snowflake Data Warehouse Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuri...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will be at the forefront of designing, building, and configuring applications. Your typical day will involve collaborating with various stakeholders to gather requirements, overseeing the development process, and ensuring that the applications meet the specified needs. You will also be responsible for troubleshooting issues and provi...