Home
Jobs

3895 Pyspark Jobs - Page 34

Filter Interviews
Min: 0 years
Max: 25 years
Min: β‚Ή0
Max: β‚Ή10000000
Setup a job Alert
Filter
JobPe aggregates results for easy application access, but you actually apply on the job portal directly.

5.0 years

0 Lacs

Chennai, Tamil Nadu, India

On-site

Linkedin logo

Greetings from TCS! TCS is hiring for ETL Tester Required Skill Set: SAS+ETL Testing Desired Experience Range: 5 to 10 Years Job Location: PAN INDIA Should be strong in Azure and ETL Testing(Highly Importance), SQL and good Knowledge in Data Warehousing (DWH) Concepts Able to work individually and meet the testing delivery expectation from End to End. Able to analyze the requirement, pro-actively identify the scenarios, co-ordinate with business team and get it clarified. Able to understand, convert and verify the business transformation logic into technical terms Should be willing and ready to put in additional effort to learn SAS Should be willing and ready to put in additional effort to learn Python and Pyspark Show more Show less

Posted 6 days ago

Apply

0 years

0 Lacs

Mumbai, Maharashtra, India

Remote

Linkedin logo

Role: Database Engineer Location : Remote Skills and Experience ● Bachelor's degree in Computer Science, Information Systems, or a related field is desirable but not essential. ● Experience with data warehousing concepts and tools (e.g., Snowflake, Redshift) to support advanced analytics and reporting, aligning with the team’s data presentation goals. ● Skills in working with APIs for data ingestion or connecting third-party systems, which could streamline data acquisition processes. ● Proficiency with tools like Prometheus, Grafana, or ELK Stack for real-time database monitoring and health checks beyond basic troubleshooting. ● Familiarity with continuous integration/continuous deployment (CI/CD) tools (e.g., Jenkins, GitHub Actions). ● Deeper expertise in cloud platforms (e.g., AWS Lambda, GCP Dataflow) for serverless data processing or orchestration. ● Knowledge of database development and administration concepts, especially with relational databases like PostgreSQL and MySQL. ● Knowledge of Python programming, including data manipulation, automation, and object-oriented programming (OOP), with experience in modules such as Pandas, SQLAlchemy, gspread, PyDrive, and PySpark. ● Knowledge of SQL and understanding of database design principles, normalization, and indexing. ● Knowledge of data migration, ETL (Extract, Transform, Load) processes, or integrating data from various sources. ● Knowledge of cloud-based databases, such as AWS RDS and Google BigQuery. ● Eagerness to develop import workflows and scripts to automate data import processes. ● Knowledge of data security best practices, including access controls, encryption, and compliance standards. ● Strong problem-solving and analytical skills with attention to detail. ● Creative and critical thinking. ● Strong willingness to learn and expand knowledge in data engineering. ● Familiarity with Agile development methodologies is a plus. ● Experience with version control systems, such as Git, for collaborative development. ● Ability to thrive in a fast-paced environment with rapidly changing priorities. ● Ability to work collaboratively in a team environment. ● Good and effective communication skills. ● Comfortable with autonomy and ability to work independently. Show more Show less

Posted 6 days ago

Apply

0 years

0 Lacs

Hyderabad, Telangana, India

On-site

Linkedin logo

Role : Palantir Tech Lead Location : Hyderabad( 5 days work from office) Skills: Python, Pyspark and Palantir. Tasks and Responsibilities: Leads data engineering activities on moderate to complex data and analytics-centric problems which have broad impact and require in-depth analysis to obtain desired results; assemble, enhance, maintain, and optimize current, enable cost savings and meet individual project or enterprise maturity objectives. advanced working knowledge of SQL, Python, and PySpark Experience using tools such as: Git/Bitbucket, Jenkins/CodeBuild, CodePipeline Experience with platform monitoring and alerts tools Work closely with Subject Matter Experts (SMEs) to design and develop Foundry front end applications with the ontology (data model) and data pipelines supporting the applications Implement data transformations to derive new datasets or create Foundry Ontology Objects necessary for business applications Implement operational applications using Foundry Tools (Workshop, Map, and/or Slate) Actively participate in agile/scrum ceremonies (stand ups, planning, retrospectives, etc.) Create and maintain documentation describing data catalog and data objects Maintain applications as usage grows and requirements change Promote a continuous improvement mindset by engaging in after action reviews and sharing learnings Use communication skills, especially for explaining technical concepts to nontechnical business leader Show more Show less

Posted 6 days ago

Apply

5.0 years

0 Lacs

Chennai, Tamil Nadu, India

On-site

Linkedin logo

Greetings from TCS! TCS is hiring for Big Data (PySpark & Scala) Location: - Chennai Desired Experience Range: 5 + Years Must-Have β€’ PySpark β€’ Hive Good-to-Have β€’ Spark β€’ HBase β€’ DQ tool β€’ Agile Scrum experience β€’ Exposure in data ingestion from disparate sources onto Big Data platform Thanks Anshika Show more Show less

Posted 6 days ago

Apply

0.0 years

0 Lacs

Chennai, Tamil Nadu

On-site

Indeed logo

IT Full-Time Job ID: DGC00714 Chennai, Tamil Nadu 3-5 Yrs β‚Ή3.5 - β‚Ή06 Yearly Job description We are hiring for a full-time position - a Python developer for designing, developing, and maintaining software applications using Python We seek immediate joiners who can work in person from our Chennai office We seek immediate joiners who can work in person from our Chennai office You will work with the Global engineering team and with the latest Data Science tools and build next-generation products Key Responsibilities: 1. Design, develop, and maintain software applications using Python. 2. Collaborate with cross-functional teams to define, design, and ship new features. 3. Participate in code and design reviews to maintain code quality standards. 4. Troubleshoot and debug issues in existing applications. Qualifications: 1. Bachelors degree in Computer Science or Proven experience as a Python Developer. 2. Strong knowledge of Python and any of the libraries such as NumPy, Pandas, and Django. 3. Experience in Pyspark. 4. Familiarity with web development frameworks such as Flask or Django. 5. Experience with cloud-based platforms such as AWS or Azure

Posted 6 days ago

Apply

0 years

0 Lacs

Gurugram, Haryana, India

On-site

Linkedin logo

Roles And Responsibilities Proficiency in building highly scalable ETL and streaming-based data pipelines using Google Cloud Platform (GCP) services and products like Biquark, Cloud Dataflow Proficiency in large scale data platforms and data processing systems such as Google Big Query, Amazon Redshift, Azure Data Lake Excellent Python, PySpark and SQL development and debugging skills, exposure to other Big Data frameworks like Hadoop Hive would be added advantage Experience building systems to retrieve and aggregate data from event-driven messaging frameworks (e.g. RabbitMQ and Pub/Sub) Secondary Skills : Cloud Big Table, AI/ML solutions, Compute Engine, Cloud Fusion (ref:hirist.tech) Show more Show less

Posted 1 week ago

Apply

4.0 years

0 Lacs

Bengaluru, Karnataka, India

On-site

Linkedin logo

Job Title: Senior Data Scientist Location: Bangalore Job Title: Assistant Manager - Security Engineering Location: UniOps Bangalore About Unilever Be part of the world’s most successful, purpose-led business. Work with brands that are well-loved around the world, that improve the lives of our consumers and the communities around us. We promote innovation, big and small, to make our business win and grow; and we believe in business as a force for good. Unleash your curiosity, challenge ideas and disrupt processes; use your energy to make this happen. Our brilliant business leaders and colleagues provide mentorship and inspiration, so you can be at your best. Every day, nine out of ten Indian households use our products to feel good, look good and get more out of life – giving us a unique opportunity to build a brighter future. Every individual here can bring their purpose to life through their work. Join us and you’ll be surrounded by inspiring leaders and supportive peers. Among them, you’ll channel your purpose, bring fresh ideas to the table, and simply be you. As you work to make a real impact on the business and the world, we’ll work to help you become a better you. About Uniops Unilever Operations (UniOps) is the global technology and operations engine of Unilever offering business services, technology, and enterprise solutions. UniOps serves over 190 locations and through a network of specialized service lines and partners delivers insights and innovations, user experiences and end-to-end seamless delivery making Unilever Purpose Led and Future Fit Background For Unilever to remain competitive in the future, the business needs to continue on the path to become data intelligent. The Data & Analytics team will persevere to make Unilever Data Intelligent, powering key decisions with data, insights, advanced analytics and AI. Our ambition is to enable democratization of data, information and insights as a completely agile organization that builds fantastic careers for our people and is accountable for delivering great work that maximizes impact and delivers growth. This Data & Analytics function endeavours to create clear accountability for all aspects of Data Strategy, Data Management, Information Management, Analytics, and Insights. We are accountable for impact of solutions, maintaining market relevance and minimising unnecessary overlaps in analytics products, ensuring simplicity and that our solutions better meet the needs of our users. We partner with the Digital and Data Legal Counsel to ensure that our Data Defence (Privacy, Governance, Quality, etc) is well structured and sufficiently robust to use data and AI correctly throughout the enterprise. We democratize information across the business, while supporting the culture shift required for data driven decision making. Our Vision Is To Make Unilever Data Intelligent, Partnering With The Business To Power Key Decisions With Data, Advanced Analytics And AI To Accelerate Growth. Our 5 Strategies To Achieve This Are Accelerate & simplify access to relevant data, information and insights Build in-house, leading-edge data, information, insights & analytics capability Lead the data & insights culture and careers to empower employees across Unilever Rapidly embed analytics products, solutions and services to drive growth Advance Information Automation at Scale The Senior Data Scientist is an exciting role in the Data Foundation. This team builds state of the art machine learning algorithms, maximising the impact of analytic solutions in driving enterprise performance. Typical initiatives include optimizing trade promotion investments, accurately forecasting customer demand, using NLP to glean insight on consumer trends from search data, and making individual assortment recommendations for each of the millions of stores that sell Unilever products. Main Purpose Of The Job The Senior Data Scientist improves business performance in the functional area of Unilever they serve, through the application of world class data science capability. They own delivery of data science on moderate projects or specific modules of a major global initiative. Key Accountabilities Interact with relevant teams to identify business challenges where data science can help Apply comprehensive data science knowledge to propose optimal techniques for key business challenges Create detailed data science proposals and project plans, flagging any limitations of proposed solution Design and prototype experimental solutions, particularly machine learning models Design scaled solutions and ensure high quality and timely delivery Facilitate industrialization and ongoing operation of solutions through well organised code, clear documentation and collaboration with ML Ops resources Govern the work of 3rd party vendors where needed to support delivery, while maximising creation of Unilever IP Represent Data Science in cross-functional governance of projects, engaging with stakeholders up to Director level Highlight recent developments in data science capability which could solve additional challenges Lead a team of up 1-2 data scientists / interns, providing career mentorship and line management Provide technical guidance to data scientists across D&A, particularly on the projects you lead Support the growth of D&A’s data science capability by contributing to activities such as tool and vendor selection, best practice definition, recruitment, and creation of training materials Build the reputation of D&A’s data science capability within Unilever and externally, through activities such as community engagement (e.g. Yammer), publications or blogs Provide ad-hoc & immediate support to the business when needed (for example Covid-19 crisis support) Depending on the specific project, the Senior Data Scientist can expect 60-90% of their work to be hands-on prototyping solutions, with the remainder spent planning and designing, overseeing and reviewing work of project staff, interfacing with stakeholders and managing team members. Experience And Qualifications Required Standards of Leadership Required in This Role Personal Mastery (Data-science and advanced analytics) Agility Business acumen Passion for High Performance Key Skills Required Professional Skills Machine learning - Expert Statistical modelling - Expert Forecasting - Expert Optimisation techniques and tools - Fully Operational Python coding - Fully Operational Data science platform tools e.g. MS Azure, Databricks - Fully Operational Deep learning (and applications to NLP & Computer Vision) - Fully Operational Collaborative development using Git repos - Fully Operational Automated Machine Learning platforms - Foundational knowledge While a broad data science technical background is required, the role will benefit from deeper skills (for example graduate studies or prior work experience) in one of the following areas, optimization, simulation, forecasting, natural language processing, computer vision or geospatial analysis. General Skills Project Management - Expert Communication / presentation skills - Expert 3rd party resource management - Expert CPG Industry analytics - Expert Strong communication and stakeholder engagement skills are essential, including the ability to influence peers and senior business stakeholders across Unilever. Relevant Experience Minimum of B.E. in a relevant technical field (e.g. Computer Science, Engineering, Statistics, Operations Research); preferably a postgraduate (Masters or Doctorate) degree At least 4 years building data science solutions to solve business problems, preferably in the CPG industry (less experience may be acceptable if balanced by strong post-grad qualifications) Experience with open source languages (eg. Python) and preferably with distributed computing (PySpark) Experience deploying solutions in a modern cloud-based architecture Experience managing the work of team members and 3rd party resource vendors Experience presenting insights and influencing decisions of senior non-technical stakeholders Key interfaces Internal Unilever operational, marketing, customer development, supply chain, product & finance teams Internal D&A teams (Engagement teams; Data CoE; Solution Factory; BDL Factory; Information Factory; Tech Transformation) Wider Unilever analytics and data science professionals External 3rd party Data Science vendors Universities Industry bodies At HUL, we believe that every individual irrespective of their race, colour, religion, gender, sexual orientation, gender identity or expression, age, nationality, caste, disability or marital status can bring their purpose to life. So apply to us, to unleash your curiosity, challenge ideas and disrupt processes; use your energy to make the world a better place. As you work to make a real impact on the business and the world, we’ll work to help you become a better you! Show more Show less

Posted 1 week ago

Apply

3.0 years

0 Lacs

Chennai, Tamil Nadu, India

On-site

Linkedin logo

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : PySpark Good to have skills : NA Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements. You will play a crucial role in developing innovative solutions to enhance business operations and efficiency. Roles & Responsibilities: - Expected to perform independently and become an SME. - Required active participation/contribution in team discussions. - Contribute in providing solutions to work related problems. - Develop and implement efficient PySpark applications. - Collaborate with team members to analyze and address application requirements. - Troubleshoot and resolve technical issues in application development. - Stay updated with the latest trends and technologies in PySpark. - Provide technical guidance and support to junior team members. Professional & Technical Skills: - Must To Have Skills: Proficiency in PySpark. - Strong understanding of data processing and manipulation using PySpark. - Experience in building scalable and efficient PySpark applications. - Knowledge of PySpark libraries and frameworks. - Good To Have Skills: Experience with data visualization tools. Additional Information: - The candidate should have a minimum of 3 years of experience in PySpark. - This position is based at our Chennai office. - A 15 years full time education is required. Show more Show less

Posted 1 week ago

Apply

0 years

0 Lacs

Pune, Maharashtra, India

On-site

Linkedin logo

Join us as a Support Analyst at Barclays, responsible for supporting the successful delivery of Location Strategy projects to plan, budget, agreed quality and governance standards. You'll spearhead the evolution of our digital landscape, driving innovation and excellence. You will harness cutting-edge technology to revolutionise our digital offerings, ensuring unparalleled customer experiences. To be successful as a Support Analyst you should have experience with: Bachelor’s degree in computers/ IT or equivalent. ITIL Process awareness with support background preferred. Good Knowledge on AWS Data Analytics Technology Stack (Glue, S3, Lambda, Lake formation, Athena) Experience in using Orchestration tools such as Apache Airflow or Snowflake Tasks. Hands on Experience in maintaining and Supporting applications on AWS Cloud. Hands on experience in pyspark, Dataframes, RDD and SparkSQL Experience in UNIX and shell scripting Experience in analysing SQL. Exposure to data governance or lineage tools such as Immuta and Alation is added advantage. Additional Skills ETL Tools exposure with Real-time and large data volumes handling and processing. Experience is supporting critical services with escalation matrix handling and customer communication. Knowledge on Ab Initio ETL tool is a plus Exposure to Automation and Tooling. You may be assessed on the key critical skills relevant for success in role, such as risk and controls, change and transformation, business acumen strategic thinking and digital and technology, as well as job-specific technical skills. This role is based in Pune. Purpose of the role To effectively monitor and maintain the bank’s critical technology infrastructure and resolve more complex technical issues, whilst minimising disruption to operations. Accountabilities Provision of technical support for the service management function to resolve more complex issues for a specific client of group of clients. Develop the support model and service offering to improve the service to customers and stakeholders. Execution of preventative maintenance tasks on hardware and software and utilisation of monitoring tools/metrics to identify, prevent and address potential issues and ensure optimal performance. Maintenance of a knowledge base containing detailed documentation of resolved cases for future reference, self-service opportunities and knowledge sharing. Analysis of system logs, error messages and user reports to identify the root causes of hardware, software and network issues, and providing a resolution to these issues by fixing or replacing faulty hardware components, reinstalling software, or applying configuration changes. Automation, monitoring enhancements, capacity management, resiliency, business continuity management, front office specific support and stakeholder management. Identification and remediation or raising, through appropriate process, of potential service impacting risks and issues. Proactively assess support activities implementing automations where appropriate to maintain stability and drive efficiency. Actively tune monitoring tools, thresholds, and alerting to ensure issues are known when they occur. Analyst Expectations To perform prescribed activities in a timely manner and to a high standard consistently driving continuous improvement. Requires in-depth technical knowledge and experience in their assigned area of expertise Thorough understanding of the underlying principles and concepts within the area of expertise They lead and supervise a team, guiding and supporting professional development, allocating work requirements and coordinating team resources. If the position has leadership responsibilities, People Leaders are expected to demonstrate a clear set of leadership behaviours to create an environment for colleagues to thrive and deliver to a consistently excellent standard. The four LEAD behaviours are: L – Listen and be authentic, E – Energise and inspire, A – Align across the enterprise, D – Develop others. OR for an individual contributor, they develop technical expertise in work area, acting as an advisor where appropriate. Will have an impact on the work of related teams within the area. Partner with other functions and business areas. Takes responsibility for end results of a team’s operational processing and activities. Escalate breaches of policies / procedure appropriately. Take responsibility for embedding new policies/ procedures adopted due to risk mitigation. Advise and influence decision making within own area of expertise. Take ownership for managing risk and strengthening controls in relation to the work you own or contribute to. Deliver your work and areas of responsibility in line with relevant rules, regulation and codes of conduct. Maintain and continually build an understanding of how own sub-function integrates with function, alongside knowledge of the organisations products, services and processes within the function. Demonstrate understanding of how areas coordinate and contribute to the achievement of the objectives of the organisation sub-function. Make evaluative judgements based on the analysis of factual information, paying attention to detail. Resolve problems by identifying and selecting solutions through the application of acquired technical experience and will be guided by precedents. Guide and persuade team members and communicate complex / sensitive information. Act as contact point for stakeholders outside of the immediate function, while building a network of contacts outside team and external to the organisation. All colleagues will be expected to demonstrate the Barclays Values of Respect, Integrity, Service, Excellence and Stewardship – our moral compass, helping us do what we believe is right. They will also be expected to demonstrate the Barclays Mindset – to Empower, Challenge and Drive – the operating manual for how we behave. Back to nav Share job X(Opens in new tab or window) Facebook(Opens in new tab or window) LinkedIn(Opens in new tab or window) Show more Show less

Posted 1 week ago

Apply

3.0 - 7.0 years

0 - 0 Lacs

Hyderabad

Work from Office

Naukri logo

Experience Required: 3+ years Technical knowledge: AWS, Python, SQL, S3, EC2, Glue, Athena, Lambda, DynamoDB, RedShift, Step Functions, Cloud Formation, CI/CD Pipelines, Github, EMR, RDS,AWS Lake Formation, GitLab, Jenkins and AWS CodePipeline. Role Summary: As a Senior Data Engineer,with over 3 years of expertise in Python, PySpark, SQL to design, develop and optimize complex data pipelines, support data modeling, and contribute to the architecture that supports big data processing and analytics to cutting-edge cloud solutions that drive business growth. You will lead the design and implementation of scalable, high-performance data solutions on AWS and mentor junior team members.This role demands a deep understanding of AWS services, big data tools, and complex architectures to support large-scale data processing and advanced analytics. Key Responsibilities: Design and develop robust, scalable data pipelines using AWS services, Python, PySpark, and SQL that integrate seamlessly with the broader data and product ecosystem. Lead the migration of legacy data warehouses and data marts to AWS cloud-based data lake and data warehouse solutions. Optimize data processing and storage for performance and cost. Implement data security and compliance best practices, in collaboration with the IT security team. Build flexible and scalable systems to handle the growing demands of real-time analytics and big data processing. Work closely with data scientists and analysts to support their data needs and assist in building complex queries and data analysis pipelines. Collaborate with cross-functional teams to understand their data needs and translate them into technical requirements. Continuously evaluate new technologies and AWS services to enhance data capabilities and performance. Create and maintain comprehensive documentation of data pipelines, architectures, and workflows. Participate in code reviews and ensure that all solutions are aligned to pre-defined architectural specifications. Present findings to executive leadership and recommend data-driven strategies for business growth. Communicate effectively with different levels of management to gather use cases/requirements and provide designs that cater to those stakeholders. Handle clients in multiple industries at the same time, balancing their unique needs. Provide mentoring and guidance to junior data engineers and team members. Requirements: 3+ years of experience in a data engineering role with a strong focus on AWS, Python, PySpark, Hive, and SQL. Proven experience in designing and delivering large-scale data warehousing and data processing solutions. Lead the design and implementation of complex, scalable data pipelines using AWS services such as S3, EC2, EMR, RDS, Redshift, Glue, Lambda, Athena, and AWS Lake Formation. Bachelor's or Masters degree in Computer Science, Engineering, or a related technical field. Deep knowledge of big data technologies and ETL tools, such as Apache Spark, PySpark, Hadoop, Kafka, and Spark Streaming. Implement data architecture patterns, including event-driven pipelines, Lambda architectures, and data lakes. Incorporate modern tools like Databricks, Airflow, and Terraform for orchestration and infrastructure as code. Implement CI/CD using GitLab, Jenkins, and AWS CodePipeline. Ensure data security, governance, and compliance by leveraging tools such as IAM, KMS, and AWS CloudTrail. Mentor junior engineers, fostering a culture of continuous learning and improvement. Excellent problem-solving and analytical skills, with a strategic mindset. Strong communication and leadership skills, with the ability to influence stakeholders at all levels. Ability to work independently as well as part of a team in a fast-paced environment. Advanced data visualization skills and the ability to present complex data in a clear and concise manner. Excellent communication skills, both written and verbal, to collaborate effectively across teams and levels. Preferred Skills: Experience with Databricks, Snowflake, and machine learning pipelines. Exposure to real-time data streaming technologies and architectures. Familiarity with containerization and serverless computing (Docker, Kubernetes, AWS Lambda).

Posted 1 week ago

Apply

10.0 years

0 Lacs

India

Remote

Linkedin logo

Role: Senior Azure / Data Engineer with (ETL/ Data warehouse background) Location: Remote, India Duration: Long Term Contract Need with 10+ years of experience Must have Skills : β€’ Min 5 years of experience in modern data engineering/data warehousing/data lakes technologies on cloud platforms like Azure, AWS, GCP, Data Bricks, etc. Azure experience is preferred over other cloud platforms. β€’ 10 + years of proven experience with SQL, schema design, and dimensional data modeling β€’ Solid knowledge of data warehouse best practices, development standards, and methodologies β€’ Experience with ETL/ELT tools like ADF, Informatica, Talend, etc., and data warehousing technologies like Azure Synapse, Azure SQL, Amazon Redshift, Snowflake, Google Big Query, etc.. β€’ Strong experience with big data tools(Databricks, Spark, etc..) and programming skills in PySpark and Spark SQL. β€’ Be an independent self-learner with a β€œlet’s get this done” approach and the ability to work in Fast paced and Dynamic environment. β€’ Excellent communication and teamwork abilities. Nice-to-Have Skills: β€’ Event Hub, IOT Hub, Azure Stream Analytics, Azure Analysis Service, Cosmo DB knowledge. β€’ SAP ECC /S/4 and Hana knowledge. β€’ Intermediate knowledge on Power BI β€’ Azure DevOps and CI/CD deployments, Cloud migration methodologies and processes Show more Show less

Posted 1 week ago

Apply

4.0 - 8.0 years

12 - 18 Lacs

Hyderabad, Chennai, Coimbatore

Hybrid

Naukri logo

We are seeking a skilled and motivated Data Engineer to join our dynamic team. The ideal candidate will have experience in designing, developing, and maintaining scalable data pipelines and architectures using Hadoop, PySpark, ETL processes , and Cloud technologies . Responsibilities: Design, develop, and maintain data pipelines for processing large-scale datasets. Build efficient ETL workflows to transform and integrate data from multiple sources. Develop and optimize Hadoop and PySpark applications for data processing. Ensure data quality, governance, and security standards are met across systems. Implement and manage Cloud-based data solutions (AWS, Azure, or GCP). Collaborate with data scientists and analysts to support business intelligence initiatives. Troubleshoot performance issues and optimize query executions in big data environments. Stay updated with industry trends and advancements in big data and cloud technologies . Required Skills: Strong programming skills in Python, Scala, or Java . Hands-on experience with Hadoop ecosystem (HDFS, Hive, Spark, etc.). Expertise in PySpark for distributed data processing. Proficiency in ETL tools and workflows (SSIS, Apache Nifi, or custom pipelines). Experience with Cloud platforms (AWS, Azure, GCP) and their data-related services. Knowledge of SQL and NoSQL databases. Familiarity with data warehousing concepts and data modeling techniques. Strong analytical and problem-solving skills. Interested can reach us at +91 7305206696/ saranyadevib@talentien.com

Posted 1 week ago

Apply

0 years

0 Lacs

Hyderabad, Telangana, India

On-site

Linkedin logo

Overview The Data Impact Analyst is part of the Data Impact team. The purpose of this team is to deliver business impact through data and industry leading analytics, all in close collaboration with the NWE commercial departments (DX, Sales and Marketing) and Europe sector teams/CoEs (Advanced Analytics, Data & Analytics, Digital, Reporting & Insights, Perfect Store). The associate will support PepsiCo NWE Commercial Data & Analytics strategy definition. He/she will own and maintain the commercial reporting landscape and play an instrumental role in data democratization, making sure that data and insights are available to everyone in an easy and effective way through standardization and new tools development (e.g. dashboarding). As a member of the Data Impact team, he/she will lead the translation of strategic business questions into analytics use cases and ultimately business impact through capturing needs, preparing relevant data sources and applying advanced analytics methods. Responsibilities Co-own data management strategy - define the way we collect, store, maintain and automate commercial data sources and assess improvement potential for the existing strategy Conduct periodic data quality checks Own and maintain existing commercial reporting landscape; assess automation and harmonization potential and align with commercial stakeholders on their reporting needs Based on the assessment, transform the existing reporting into Power BI dashboards; develop new reports if needed Use tools like Python/PySpark in Azure Databricks to prepare data for analytics use cases Work with commercial teams and translate their strategic business questions into analytics use cases Act as a data and analytics evangelist, be at the forefront of data-driven models and insights and lead others to leverage data in their decision making Qualifications Excellent analytical skills with a natural ability to visualize data in a way that uncovers (new) insights Ability to process and work with large and diversified datasets Strong experience in Power BI - Backend/Frontend Previous experience with data preparation tools like Alteryx and Azure Databricks Experience in Databricks and Python/PySpark - Should be able to perform an ETL transformation of a mid-large scale dataset Previous experience with data visualization tools - preferably MS Power BI Good to have knowledge of R/understanding and running simple machine learning models/SQL but not mandatory Attention to detail, accuracy and ability to work towards tight deadlines Intellectually curious, with an interest in how analytics can be leveraged to derive business value Effective verbal and written communication skills E2E project management experience is preferable - i.e. from collecting/understanding business requirements through development to implementation and evaluation Show more Show less

Posted 1 week ago

Apply

3.0 - 8.0 years

0 - 0 Lacs

Chennai

Hybrid

Naukri logo

You Lead the Way. We've Got Your Back. With the right backing, people and businesses have the power to progress in incredible ways. When you join Team Amex, you become part of a global and diverse community of colleagues with an unwavering commitment to back our customers, communities and each other. Here, youll learn and grow as we help you create a career journey thats unique and meaningful to you with benefits, programs, and flexibility that support you personally and professionally. At American Express, you’ll be recognized for your contributions, leadership, and impactβ€”every colleague has the opportunity to share in the company’s success. Together, we’ll win as a team, striving to uphold our company values and powerful backing promise to provide the world’s best customer experience every day. And we’ll do it with the utmost integrity, and in an environment where everyone is seen, heard and feels like they belong. As part of our diverse tech team, you can architect, code and ship software that makes us an essential part of our customers’ digital lives. Here, you can work alongside talented engineers in an open, supportive, inclusive environment where your voice is valued, and you make your own decisions on what tech to use to solve challenging problems. Amex offers a range of opportunities to work with the latest technologies and encourages you to back the broader engineering community through open source. And because we understand the importance of keeping your skills fresh and relevant, we give you dedicated time to invest in your professional development. Find your place in technology on #TeamAmex. How will you make an impact in this role? Build NextGen Data Strategy, Data Virtualization, Data Lakes Warehousing Transform and improve performance of existing reporting & analytics use cases with more efficient and state of the art data engineering solutions. Analytics Development to realize advanced analytics vision and strategy in a scalable, iterative manner. Deliver software that provides superior user experiences, linking customer needs and business drivers together through innovative product engineering. Cultivate an environment of Engineering excellence and continuous improvement, leading changes that drive efficiencies into existing Engineering and delivery processes. Own accountability for all quality aspects and metrics of product portfolio, including system performance, platform availability, operational efficiency, risk management, information security, data management and cost effectiveness. Work with key stakeholders to drive Software solutions that align to strategic roadmaps, prioritized initiatives and strategic Technology directions. Work with peers, staff engineers and staff architects to assimilate new technology and delivery methods into scalable software solutions. Minimum Qualifications: Bachelor’s degree in computer science, Computer Science Engineering, or related field required; Advanced Degree preferred. 3- 12 years of hands-on experience in implementing large data-warehousing projects, strong knowledge of latest NextGen BI & Data Strategy & BI Tools Proven experience in Business Intelligence, Reporting on large datasets, Data Virtualization Tools, Big Data, GCP, JAVA, Microservices Strong systems integration architecture skills and a high degree of technical expertise, ranging across a number of technologies with a proven track record of turning new technologies into business solutions. Should be good in one programming language python/Java. Should have good understanding of data structures. GCP /cloud knowledge has added advantage. PowerBI, Tableau and looker good knowledge and understanding. Outstanding influential and collaboration skills; ability to drive consensus and tangible outcomes, demonstrated by breaking down silos and fostering cross communication process. Experience managing in a fast paced, complex, and dynamic global environment. Outstanding influential and collaboration skills; ability to drive consensus and tangible outcomes, demonstrated by breaking down silos and fostering cross communication process. Preferred Qualifications: Bachelor’s degree in computer science, Computer Science Engineering, or related field required; Advanced Degree preferred. 5+ years of hands-on experience in implementing large data-warehousing projects, strong knowledge of latest NextGen BI & Data Strategy & BI Tools Proven experience in Business Intelligence, Reporting on large datasets, Oracle Business Intelligence (OBIEE), Tableau, MicroStrategy, Data Virtualization Tools, Oracle PL/SQL, Informatica, Other ETL Tools like Talend, Java Should be good in one programming language python/Java. Should be good data structures and reasoning. GCP knowledge has added advantage or cloud knowledge. PowerBI, Tableau and looker good knowledge and understanding. Strong systems integration architecture skills and a high degree of technical expertise, ranging across several technologies with a proven track record of turning new technologies into business solutions. Outstanding influential and collaboration skills; ability to drive consensus and tangible outcomes, demonstrated by breaking down silos and fostering cross communication process. Compliance Language We back you with benefits that support your holistic well-being so you can be and deliver your best. This means caring for you and your loved ones' physical, financial, and mental health, as well as providing the flexibility you need to thrive personally and professionally: Competitive base salaries Bonus incentives Support for financial-well-being and retirement Comprehensive medical, dental, vision, life insurance, and disability benefits (depending on location) Flexible working model with hybrid, onsite or virtual arrangements depending on role and business need Generous paid parental leave policies (depending on your location) Free access to global on-site wellness centers staffed with nurses and doctors (depending on location) Free and confidential counseling support through our Healthy Minds program Career development and training opportunities American Express is an equal opportunity employer and makes employment decisions without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran status, disability status, age, or any other status protected by law. Offer of employment with American Express is conditioned upon the successful completion of a background verification check, subject to applicable laws and regulations.

Posted 1 week ago

Apply

8.0 years

0 Lacs

India

Remote

Linkedin logo

Data Architect Long Term Contract (Initially 6 months rolling) - Fully Remote $25 - $30 per hour ($4000 - $4800) Start Date: ASAP (must be available within 30 days of offer) KEY SKILLS An experienced Data Architect to build robust data architecture solutions using a range of cutting-edge technologies. Must have a minimum of 8-12 years Big Data Experience Strong and recent PySpark (5+ years) & Python Any experience of Palantir Foundry or similar products such as Databricks would be highly advantageous. Show more Show less

Posted 1 week ago

Apply

0 years

0 Lacs

Chennai, Tamil Nadu, India

On-site

Linkedin logo

Transform data into a format that can be easily analyzed by developing, maintaining, and testing infrastructures for data generation. Work closely with data scientists and are largely in charge of architecting solutions for data scientists that enable them to do their jobs. Role involves creating Data pipelines and integrating, transforming & enabling data for wider enterprise use. Job Description Duties for this role include but not limited to: supporting the design, build, test and maintain data pipelines at big data scale. Assists with updating data from multiple data sources. Work on batch processing of collected data and match its format to the stored data, make sure that the data is ready to be processed and analyzed. Assisting with keeping the ecosystem and the pipeline optimized and efficient, troubleshooting standard performance, data related problems and provide L3 support. Implementing parsers, validators, transformers and correlators to reformat, update and enhance the data. Provides recommendations to highly complex problems. Providing guidance to those in less senior positions. Additional Job Description Data Engineers play a pivotal role within Dataworks, focused on creating and driving engineering innovation and facilitating the delivery of key business initiatives. Acting as a β€œuniversal translator” between IT, business, software engineers and data scientists, data engineers collaborate across multi-disciplinary teams to deliver value. Data Engineers will work on those aspects of the Dataworks platform that govern the ingestion, transformation, and pipelining of data assets, both to end users within FedEx and into data products and services that may be externally facing. Day-to-day, they will be deeply involved in code reviews and large-scale deployments. Essential Job Duties & Responsibilities Understanding in depth both the business and technical problems Dataworks aims to solve Building tools, platforms and pipelines to enable teams to clearly and cleanly analyze data, build models and drive decisions Scaling up from β€œlaptop-scale” to β€œcluster scale” problems, in terms of both infrastructure and problem structure and technique Collaborating across teams to drive the generation of data driven operational insights that translate to high value optimized solutions. Delivering tangible value very rapidly, collaborating with diverse teams of varying backgrounds and disciplines Codifying best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases Interacting with senior technologists from the broader enterprise and outside of FedEx (partner ecosystems and customers) to create synergies and ensure smooth deployments to downstream operational systems Skill/Knowledge Considered a Plus Technical background in computer science, software engineering, database systems, distributed systems Fluency with distributed and cloud environments and a deep understanding of optimizing computational considerations with theoretical properties Experience in building robust cloud-based data engineering and curation solutions to create data products useful for numerous applications Detailed knowledge of the Microsoft Azure tooling for large-scale data engineering efforts and deployments is highly preferred. Experience with any combination of the following azure tools: Azure Databricks, Azure Data Factory, Azure SQL D, Azure Synapse Analytics Developing and operationalizing capabilities and solutions including under near real-time high-volume streaming conditions. Hands-on development skills with the ability to work at the code level and help debug hard to resolve issues. A compelling track record of designing and deploying large scale technical solutions, which deliver tangible, ongoing value Direct experience having built and deployed robust, complex production systems that implement modern, data processing methods at scale Ability to context-switch, to provide support to dispersed teams which may need an β€œexpert hacker” to unblock an especially challenging technical obstacle, and to work through problems as they are still being defined Demonstrated ability to deliver technical projects with a team, often working under tight time constraints to deliver value An β€˜engineering’ mindset, willing to make rapid, pragmatic decisions to improve performance, accelerate progress or magnify impact Comfort with working with distributed teams on code-based deliverables, using version control systems and code reviews Ability to conduct data analysis, investigation, and lineage studies to document and enhance data quality and access Use of agile and devops practices for project and software management including continuous integration and continuous delivery Demonstrated expertise working with some of the following common languages and tools: Spark (Scala and PySpark), Kafka and other high-volume data tools SQL and NoSQL storage tools, such as MySQL, Postgres, MongoDB/CosmosDB Java, Python data tools Azure DevOps experience to track work, develop using git-integrated version control patterns, and build and utilize CI/CD pipelines Working knowledge and experience implementing data architecture patterns to support varying business needs Experience with different data types (json, xml, parquet, avro, unstructured) for both batch and streaming ingestions Use of Azure Kubernetes Services, Eventhubs, or other related technologies to implement streaming ingestions Experience developing and implementing alerting and monitoring frameworks Working knowledge of Infrastructure as Code (IaC) through Terraform to create and deploy resources Implementation experience across different data stores, messaging systems, and data processing engines Data integration through APIs and/or REST service PowerPlatform (PowerBI, PowerApp, PowerAutomate) development experience a plus Minimum Qualifications Data Engineer I: Bachelor’s Degree in Information Systems, Computer Science or a quantitative discipline such as Mathematics or Engineering and/or One (1) year equivalent formal training or work experience. Basic knowledge in data engineering and machine learning frameworks including design, development and implementation of highly complex systems and data pipelines. Basic knowledge in Information Systems including design, development and implementation of large batch or online transaction-based systems. Experience as a junior member of multi-functional project teams. Strong oral and written communication skills. A related advanced degree may offset the related experience requirements. Data Engineer II Bachelor's Degree in Computer Science, Information Systems, a related quantitative field such as Engineering or Mathematics or equivalent formal training or work experience. Two (2) years equivalent work experience in measurement and analysis, quantitative business problem solving, simulation development and/or predictive analytics. Strong knowledge in data engineering and machine learning frameworks including design, development and implementation of highly complex systems and data pipelines. Strong knowledge in Information Systems including design, development and implementation of large batch or online transaction-based systems. Strong understanding of the transportation industry, competitors, and evolving technologies. Experience as a member of multi-functional project teams. Strong oral and written communication skills. A related advanced degree may offset the related experience requirements. Data Engineer III Bachelor’s Degree in Information Systems, Computer Science or a quantitative discipline such as Mathematics or Engineering and/or equivalent formal training or work experience. Three to Four (3 - 4) years equivalent work experience in measurement and analysis, quantitative business problem solving, simulation development and/or predictive analytics. Extensive knowledge in data engineering and machine learning frameworks including design, development and implementation of highly complex systems and data pipelines. Extensive knowledge in Information Systems including design, development and implementation of large batch or online transaction-based systems. Strong understanding of the transportation industry, competitors, and evolving technologies. Experience providing leadership in a general planning or consulting setting. Experience as a senior member of multi-functional project teams. Strong oral and written communication skills. A related advanced degree may offset the related experience requirements. Data Engineer Lead Bachelor’s Degree in Information Systems, Computer Science, or a quantitative discipline such as Mathematics or Engineering and/or equivalent formal training or work experience. Five to Seven (5 - 7) years equivalent work experience in measurement and analysis, quantitative business problem solving, simulation development and/or predictive analytics. Extensive knowledge in data engineering and machine learning frameworks including design, development and implementation of highly complex systems and data pipelines. Extensive knowledge in Information Systems including design, development and implementation of large batch or online transaction-based systems. Strong understanding of the transportation industry, competitors, and evolving technologies. Experience providing leadership in a general planning or consulting setting. Experience as a leader or a senior member of multi-function project teams. Strong oral and written communication skills. A related advanced degree may offset the related experience requirements. Analytical Skills, Accuracy & Attention to Detail, Planning & Organizing Skills, Influencing & Persuasion Skills, Presentation Skills FedEx was built on a philosophy that puts people first, one we take seriously. We are an equal opportunity/affirmative action employer and we are committed to a diverse, equitable, and inclusive workforce in which we enforce fair treatment, and provide growth opportunities for everyone. All qualified applicants will receive consideration for employment regardless of age, race, color, national origin, genetics, religion, gender, marital status, pregnancy (including childbirth or a related medical condition), physical or mental disability, or any other characteristic protected by applicable laws, regulations, and ordinances. Our Company FedEx is one of the world's largest express transportation companies and has consistently been selected as one of the top 10 World’s Most Admired Companies by "Fortune" magazine. Every day FedEx delivers for its customers with transportation and business solutions, serving more than 220 countries and territories around the globe. We can serve this global network due to our outstanding team of FedEx team members, who are tasked with making every FedEx experience outstanding. Our Philosophy The People-Service-Profit philosophy (P-S-P) describes the principles that govern every FedEx decision, policy, or activity. FedEx takes care of our people; they, in turn, deliver the impeccable service demanded by our customers, who reward us with the profitability necessary to secure our future. The essential element in making the People-Service-Profit philosophy such a positive force for the company is where we close the circle, and return these profits back into the business, and invest back in our people. Our success in the industry is attributed to our people. Through our P-S-P philosophy, we have a work environment that encourages team members to be innovative in delivering the highest possible quality of service to our customers. We care for their well-being, and value their contributions to the company. Our Culture Our culture is important for many reasons, and we intentionally bring it to life through our behaviors, actions, and activities in every part of the world. The FedEx culture and values have been a cornerstone of our success and growth since we began in the early 1970’s. While other companies can copy our systems, infrastructure, and processes, our culture makes us unique and is often a differentiating factor as we compete and grow in today’s global marketplace. Show more Show less

Posted 1 week ago

Apply

2.0 - 3.0 years

0 Lacs

Pune, Maharashtra, India

On-site

Linkedin logo

Job Title: Python Developer Role Overview: We are seeking a motivated Python Developer to join our dynamic team. The ideal candidate will have 2-3 years of experience in developing scalable applications and be proficient in Python. You will work on creating, enhancing, and maintaining innovative software solutions while collaborating with cross-functional teams to deliver exceptional results Requirements Key Responsibilities: β€’ Design and develop scalable backend services and RESTful APIs using Flask, FastAPI, or Django (DRF). β€’ Collaborate with product managers and other developers to define software requirements and mentor Jr developers. β€’ Develop robust integration with SQL (PostgreSQL preferred) and NoSQL databases like MongoDB. β€’ Optimize performance and scalability of applications handling large volumes of data. β€’ Write clean, modular, and well-documented code following best practices. β€’ Contribute to architectural decisions and peer code reviews. β€’ Apply best practices in containerization (Docker), and CI/CD pipelines (preferred but not mandatory). β€’ Contribute to projects in Generative AI (GenAI) or Data Engineering depending on team needs and your expertise. Required Skills & Qualifications: β€’ 2–3 years of professional experience in Python backend development. β€’ Strong understanding of Flask; experience with FastAPI, Django, or DRF is a plus. β€’ Solid experience working with PostgreSQL and MongoDB. β€’ Familiarity with REST API development and JSON-based communication. β€’ Experience in building and optimizing scalable web services or microservices. β€’ Experience with redis and celery. β€’ Hands-on experience with version control systems such as Git. β€’ Exposure to GenAI (e.g., llama, OpenAI, HuggingFace, LangChain, Agentic AI etc) or Data Engineering (e.g., ETL pipelines, pandas, numpy) is a strong advantage. Good to Have: β€’ Experience working with Docker and cloud platforms (AWS/GCP/Azure). β€’ Familiarity with tools like Pyspark β€’ Familiarity with ML model serving. Benefits Benefits Why Join Us? Work on cutting-edge technologies and impactful projects. Opportunities for career growth and development. Collaborative and inclusive work environment. Competitive salary and benefits package. Show more Show less

Posted 1 week ago

Apply

0 years

0 Lacs

Pune, Maharashtra, India

On-site

Linkedin logo

Job Description At Rolls-Royce, we look beyond tomorrow. We continually pioneer integrated power and propulsion solutions to deliver cleaner, safer and more competitive power. Rolls-Royce Power Systems is headquartered in Friedrichshafen in southern Germany and employs around 9,000 people. The product portfolio includes MTU-brand high-speed engines and propulsion systems for ships, power generation, heavy land, rail and defense vehicles and for the oil and gas industry as well as diesel and gas systems and battery containers for mission critical, standby and continuous power, combined generation of heat and power, and micro grids. Experienced and highly motivated Project Manager to oversee the Supply, Installation, Testing & commissioning of diesel / Gas generators. The Project Manager will be responsible for ensuring the successful planning, execution, and completion of generator installation projects, ensuring adherence to budget, timeline, quality, and safety standards. The role requires coordination with multiple stakeholders, including internal teams, contractors, vendors, and clients to ensure that the power infrastructure is installed and commissioned efficiently to meet operational requirements. Work with us and we’ll welcome you into an inclusive culture, one that invests in your continuous learning and development, and gives you access to a wide breadth and depth of experience. Internship Program – Key Opportunities And Responsibilities Study literature and develop scripts and algorithm for the defined area of research Perform data and statistical analysis Develop predictive maintenance models and implement alarm systems Automation of tasks related to data logger commissioning Develop models related to reliability, Life cycle cost and reuse rate Ideal Candidate/Qualification Graduate (B.E. / B. Tech.) / postgraduate (MS / ME/ M Tech.) final year in Computer Science, IT, Electronics, Mechatronics or equivalent field. Strong hands-on Python and other coding platforms like Java and R Knowledge machine learning, artificial intelligence is advantageous, sound knowledge of statistic Experienced in cloud computing platforms, preferably Microsoft Azure and Databricks, Good Knowledge on Hadoop 2.0 ecosystem and Data structures Competent in algorithm development and optimization with respect to time and space complexity. Should have sufficient knowledge on stream processing. Working knowledge on PySpark/Spark to handle big data. Knowledge of automation using sripts. Should be able to work in an agile environment, within a self-organizing team. Collaboration and teamwork, with a willingness to share solutions and best practices across teams. Proactive in approach, Ability to apply logical, analytical, and innovative thinking on a range of technical problems. Location – Pune Internship Duration – 6 months We are an equal opportunities employer. We’re committed to developing a diverse workforce and an inclusive working environment. We believe that people from different backgrounds and cultures give us different perspectives which are crucial to innovation and problem solving. We believe the more diverse perspectives we have, the more successful we’ll be. By building a culture of caring and belonging, we give everyone who works here the opportunity to realize their full potential. You can learn more about our global Inclusion strategy at Our people | Rolls-Royce Type of Contract Temporary (Fixed Term) Show more Show less

Posted 1 week ago

Apply

3.0 years

0 Lacs

India

On-site

Linkedin logo

Coursera was launched in 2012 by Andrew Ng and Daphne Koller, with a mission to provide universal access to world-class learning. It is now one of the largest online learning platforms in the world, with 175 million registered learners as of March 31, 2025. Coursera partners with over 350 leading universities and industry leaders to offer a broad catalog of content and credentials, including courses, Specializations, Professional Certificates, and degrees. Coursera’s platform innovations enable instructors to deliver scalable, personalized, and verified learning experiences to their learners. Institutions worldwide rely on Coursera to upskill and reskill their employees, citizens, and students in high-demand fields such as GenAI, data science, technology, and business. Coursera is a Delaware public benefit corporation and a B Corp. Join us in our mission to create a world where anyone, anywhere can transform their life through access to education. We're seeking talented individuals who share our passion and drive to revolutionize the way the world learns. At Coursera, we are committed to building a globally diverse team and are thrilled to extend employment opportunities to individuals in any country where we have a legal entity. We require candidates to possess eligible working rights and have a compatible timezone overlap with their team to facilitate seamless collaboration. Coursera has a commitment to enabling flexibility and workspace choices for employees. Our interviews and onboarding are entirely virtual, providing a smooth and efficient experience for our candidates. As an employee, we enable you to select your main way of working, whether it's from home, one of our offices or hubs, or a co-working space near you. About The Role We at Coursera are seeking a highly skilled and motivated AI Specialist with expertise in developing and deploying advanced AI solutions. The ideal candidate will have 3+ years of experience, with a strong focus on leveraging AI technologies to derive insights, build predictive models, and enhance platform capabilities. This role offers a unique opportunity to contribute to cutting-edge projects that transform the online learning experience. Key Responsibilities Deploy and customize AI/ML solutions using tools and platforms from Google AI, AWS, or other providers. Develop and optimize customer journey analytics to identify actionable insights and improve user experience. Design, implement, and optimize models for predictive analytics, information extraction, semantic parsing, and topic modelling. Perform comprehensive data cleaning and preprocessing to ensure high-quality inputs for model training and deployment. Build, maintain, and refine AI pipelines for data gathering, curation, model training, evaluation, and monitoring. Analyze large-scale datasets, including customer reviews, to derive insights for improving recommendation systems and platform features. Train and support team members in adopting and managing AI-driven tools and processes. Document solutions, workflows, and troubleshooting processes to ensure knowledge continuity. Stay informed on emerging AI/ML technologies to recommend suitable solutions for new use cases. Evaluate and enhance the quality of video and audio content using AI-driven techniques. Qualifications Education: Bachelor's degree in Computer Science, Machine Learning, or a related field (required). Experience: 3+ years of experience in AI/ML development, with a focus on predictive modelling and data-driven insights. Proven experience in deploying AI solutions using platforms like Google AI, AWS, Microsoft Azure, or similar. Proficiency in programming languages such as Python, Java, or similar for AI tool customization and deployment. Strong understanding of APIs, cloud services, and integration of AI tools with existing systems. Proficiency in building and scaling AI pipelines for data engineering, model training, and monitoring. Experience with frameworks and libraries for building AI agents, such as LangChain, AutoGen Familiarity with designing autonomous workflows using LLMs and external APIs Technical Skills: Programming: Advanced proficiency in Python, PyTorch, and TensorFlow, SciKit-Learn Data Engineering: Expertise in data cleaning, preprocessing, and handling large-scale datasets. Preferred experience with tools like AWS Glue, PySpark, and AWS S3. Cloud Technologies: Experience with AWS SageMaker, Google AI, Google Vertex AI, Databricks Strong SQL skills and advanced proficiency in statistical programming languages such as Python, along with experience using data manipulation libraries (e.g., Pandas, NumPy). Coursera is an Equal Employment Opportunity Employer and considers all qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, age, marital status, national origin, protected veteran status, disability, or any other legally protected class. If you are an individual with a disability and require a reasonable accommodation to complete any part of the application process, please contact us at accommodations@coursera.org. For California Candidates, please review our CCPA Applicant Notice here. For our Global Candidates, please review our GDPR Recruitment Notice here. Show more Show less

Posted 1 week ago

Apply

2.0 - 3.0 years

6 - 7 Lacs

Pune

Work from Office

Naukri logo

Data Engineer Job Description : Jash Data Sciences: Letting Data Speak! Do you love solving real-world data problems with the latest and best techniques? And having fun while solving them in a team! Then come and join our high-energy team of passionate data people. Jash Data Sciences is the right place for you. We are a cutting-edge Data Sciences and Data Engineering startup based in Pune, India. We believe in continuous learning and evolving together. And we let the data speak! What will you be doing? You will be discovering trends in the data sets and developing algorithms to transform raw data for further analytics Create Data Pipelines to bring in data from various sources, with different formats, transform it, and finally load it to the target database. Implement ETL/ ELT processes in the cloud using tools like AirFlow, Glue, Stitch, Cloud Data Fusion, and DataFlow. Design and implement Data Lake, Data Warehouse, and Data Marts in AWS, GCP, or Azure using Redshift, BigQuery, PostgreSQL, etc. Creating efficient SQL queries and understanding query execution plans for tuning queries on engines like PostgreSQL. Performance tuning of OLAP/ OLTP databases by creating indices, tables, and views. Write Python scripts for the orchestration of data pipelines Have thoughtful discussions with customers to understand their data engineering requirements. Break complex requirements into smaller tasks for execution. What do we need from you? Strong Python coding skills with basic knowledge of algorithms/data structures and their application. Strong understanding of Data Engineering concepts including ETL, ELT, Data Lake, Data Warehousing, and Data Pipelines. Experience designing and implementing Data Lakes, Data Warehouses, and Data Marts that support terabytes of scale data. A track record of implementing Data Pipelines on public cloud environments (AWS/GCP/Azure) is highly desirable A clear understanding of Database concepts like indexing, query performance optimization, views, and various types of schemas. Hands-on SQL programming experience with knowledge of windowing functions, subqueries, and various types of joins. Experience working with Big Data technologies like PySpark/ Hadoop A good team player with the ability to communicate with clarity Show us your git repo/ blog! Qualification 1-2 years of experience working on Data Engineering projects for Data Engineer I 2-5 years of experience working on Data Engineering projects for Data Engineer II 1-5 years of Hands-on Python programming experience Bachelors/Masters' degree in Computer Science is good to have Courses or Certifications in the area of Data Engineering will be given a higher preference. Candidates who have demonstrated a drive for learning and keeping up to date with technology by continuing to do various courses/self-learning will be given high preference.

Posted 1 week ago

Apply

10.0 years

0 Lacs

Kolkata, West Bengal, India

On-site

Linkedin logo

Greetings from TCS! Role: Azure Data Engineer Location: Kolkata, Bangalore, Hyderabad Exp: 10+ Years Must-Have** 1. Azure Data Factory 2. Azure Data Bricks 3. Python 4. Sql Query writing Good-to-Have Pyspark Python SQL Query writing / optimization PowerShell Show more Show less

Posted 1 week ago

Apply

4.0 - 6.0 years

6 - 8 Lacs

Bengaluru

Work from Office

Naukri logo

Design and implement cloud-native data architectures on AWS, including data lakes, data warehouses, and streaming pipelines using services like S3, Glue, Redshift, Athena, EMR, Lake Formation, and Kinesis. Develop and orchestrate ETL/ELT pipelines Required Candidate profile Participate in pre-sales and consulting activities such as: Engaging with clients to gather requirements and propose AWS-based data engineering solutions. Supporting RFPs/RFIs, technical proposals

Posted 1 week ago

Apply

6.0 - 11.0 years

8 - 12 Lacs

Mumbai, Delhi / NCR, Bengaluru

Work from Office

Naukri logo

Senior Data Engineer (Remote, Contract 6 Months) Databricks, ADF, and PySpark. We are hiring a Senior Data Engineer for a 6-month remote contract position. The ideal candidate is highly skilled in building scalable data pipelines and working within the Azure cloud ecosystem, especially Databricks, ADF, and PySpark. You'll work closely with cross-functional teams to deliver enterprise-level data engineering solutions. KeyResponsibilities Build scalable ETL pipelines and implement robust data solutions in Azure. Manage and orchestrate workflows using ADF, Databricks, ADLS Gen2, and Key Vaults. Design and maintain secure and efficient data lake architecture. Work with stakeholders to gather data requirements and translate them into technical specs. Implement CI/CD pipelines for seamless data deployment using Azure DevOps. Monitor data quality, performance bottlenecks, and scalability issues. Write clean, organized, reusable PySpark code in an Agile environment. Document pipelines, architectures, and best practices for reuse. MustHaveSkills Experience: 6+ years in Data Engineering Tech Stack: SQL, Python, PySpark, Spark, Azure Databricks, ADF, ADLS Gen2, Azure DevOps, Key Vaults Core Expertise: Data Warehousing, ETL, Data Pipelines, Data Modelling, Data Governance Agile, SDLC, Containerization (Docker), Clean coding practices GoodToHaveSkills Event Hubs, Logic Apps Power BI Strong logic building and competitive programming background Location : - Remote, Mumbai, Delhi / NCR, Bengaluru , Kolkata, Chennai, Hyderabad, Ahmedabad, Pune

Posted 1 week ago

Apply

0 years

0 Lacs

Hyderabad, Telangana, India

On-site

Linkedin logo

Position Description Founded in 1976, CGI is among the world's largest independent IT and business consulting services firms. With 94,000 consultants and professionals globally, CGI delivers an end-to-end portfolio of capabilities, from strategic IT and business consulting to systems integration, managed IT and business process services, and intellectual property solutions. CGI works with clients through a local relationship model complemented by a global delivery network that helps clients digitally transform their organizations and accelerate results. CGI Fiscal 2024 reported revenue is CA$14.68 billion, and CGI shares are listed on the TSX (GIB.A) and the NYSE (GIB). Learn more at cgi.com. Position - Senior Software Engineer Experience - 4 - 7 Yrs Category - Software Development/Engineering Shift - 1 to 10 PM Location - BNG/HYD/CHN Position Id - J0125-0901 Work Type - Hybrid Employment Type - Full time Education - Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field. Your future duties and responsibilities We are looking for a talented Data Engg to join our team. In this role, you will develop, optimize, and maintain scalable applications, and be responsible for building efficient, testable, and reusable code. Your work will involve collaborating with cross-functional teams to deliver high-quality software that meets our clients' needs. Write reusable, testable, and efficient code. Implement security and data protection solutions. Develop and maintain robust and scalable backend systems and APIs using Python. Integrate user-facing elements developed by front-end developers with server-side logic. Work with various databases (SQL, NoSQL) to ensure efficient data storage and retrieval. Required Qualifications To Be Successful In This Role Programing Language : Python, Pyspark Bigdata Tech – Data Bricks, Spark, Hadoop, Hive Cloud – AWS Database – RDBMS & No SQL Shell Scripting Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field. CGI is an equal opportunity employer. In addition, CGI is committed to providing accommodation for people with disabilities in accordance with provincial legislation. Please let us know if you require reasonable accommodation due to a disability during any aspect of the recruitment process and we will work with you to address your needs. Together, as owners, let’s turn meaningful insights into action. Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you’ll reach your full potential because… You are invited to be an owner from day 1 as we work together to bring our Dream to life. That’s why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company’s strategy and direction. Your work creates value. You’ll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise. You’ll shape your career by joining a company built to grow and last. You’ll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons. Come join our teamβ€”one of the largest IT and business consulting services firms in the world. Show more Show less

Posted 1 week ago

Apply

6.0 years

0 Lacs

Hyderabad, Telangana, India

On-site

Linkedin logo

Position Description At CGI, we’re a team of builders. We call our employees members because all who join CGI are building their own company - one that has grown to 72,000 professionals located in 40 countries. Founded in 1976, CGI is a leading IT and business process services firm committed to helping clients succeed. We have the global resources, expertise, stability and dedicated professionals needed to achieve. At CGI, we’re a team of builders. We call our employees members because all who join CGI are building their own company - one that has grown to 72,000 professionals located in 40 countries. Founded in 1976, CGI is a leading IT and business process services firm committed to helping clients succeed. We have the global resources, expertise, stability and dedicated professionals needed to achieve results for our clients - and for our members. Come grow with us. Learn more at www.cgi.com. This is a great opportunity to join a winning team. CGI offers a competitive compensation package with opportunities for growth and professional development. Benefits for full-time, permanent members start on the first day of employment and include a paid time-off program and profit participation and stock purchase plans. We wish to thank all applicants for their interest and effort in applying for this position, however, only candidates selected for interviews will be contacted. No unsolicited agency referrals please. Job Title: Python Developer Position: Senior Software Engineer Experience: 4.6 – 6 Years Category: Software Development/ Engineering Main location: Bangalore/Hyderabad/Chennai Position ID: J1024-1413 Employment Type: Full Time Works independently under limited supervision and applies knowledge of subject matter in Applications Development. Possess sufficient knowledge and skills to effectively deal with issues, challenges within field of specialization to develop simple applications solutions. Second level professional with direct impact on results and outcome. Qualification: Bachelor's degree in Computer Science or related field or higher with minimum 4 years of relevant experience. Your future duties and responsibilities Completes the delivery of design, code or testing for modules or multiple functions related to IS development initiatives. Prepares requirement definition, design, technical specifications. Provides coding, testing and implementation support for identified technical platform (i.e., Mainframe, Mid-range, Distributed or Web) Analyzes user requirements, and defines technical project scope and assumptions for assigned tasks. Creates business and/or technical designs for new systems, and/or modifications to existing systems. Required Qualifications To Be Successful In This Role Position: Senior Software Engineer Experience: 4.6 - 6 Years Main location: Hyderabad and Bangalore Must-Have Skills: 4+ years of development experience with Python/AWS technologies. Hands on experience with Python, Pyspark, AWS and SQL AWS services required: S3, Lemda,Dynamo DB and etc.. Working experience in TDD and BDD frameworks. Provide technical direction on design considerations, including performance, scalability, availability, maintainability, and auditability. Strong customer facing experience. Propose and Design the solution approach to cater business requirements by building/enhancing re-usable components Working experience SAFe delivery model. Good organizational and written/verbal communication skills Good presentation skills Positive attitude and team focus are required Good-to-Have Skills: Experience with IBM Spectrum Conductor is an added advantage. Python utilities like interacting with FLASK API, Sharepoint API is an added advantage. Together, as owners, let’s turn meaningful insights into action. Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you’ll reach your full potential because… You are invited to be an owner from day 1 as we work together to bring our Dream to life. That’s why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company’s strategy and direction. Your work creates value. You’ll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise. You’ll shape your career by joining a company built to grow and last. You’ll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons. Come join our teamβ€”one of the largest IT and business consulting services firms in the world. Show more Show less

Posted 1 week ago

Apply

Exploring PySpark Jobs in India

PySpark, a powerful data processing framework built on top of Apache Spark and Python, is in high demand in the job market in India. With the increasing need for big data processing and analysis, companies are actively seeking professionals with PySpark skills to join their teams. If you are a job seeker looking to excel in the field of big data and analytics, exploring PySpark jobs in India could be a great career move.

Top Hiring Locations in India

Here are 5 major cities in India where companies are actively hiring for PySpark roles: 1. Bangalore 2. Pune 3. Hyderabad 4. Mumbai 5. Delhi

Average Salary Range

The estimated salary range for PySpark professionals in India varies based on experience levels. Entry-level positions can expect to earn around INR 6-8 lakhs per annum, while experienced professionals can earn upwards of INR 15 lakhs per annum.

Career Path

In the field of PySpark, a typical career progression may look like this: 1. Junior Developer 2. Data Engineer 3. Senior Developer 4. Tech Lead 5. Data Architect

Related Skills

In addition to PySpark, professionals in this field are often expected to have or develop skills in: - Python programming - Apache Spark - Big data technologies (Hadoop, Hive, etc.) - SQL - Data visualization tools (Tableau, Power BI)

Interview Questions

Here are 25 interview questions you may encounter when applying for PySpark roles:

  • Explain what PySpark is and its main features (basic)
  • What are the advantages of using PySpark over other big data processing frameworks? (medium)
  • How do you handle missing or null values in PySpark? (medium)
  • What is RDD in PySpark? (basic)
  • What is a DataFrame in PySpark and how is it different from an RDD? (medium)
  • How can you optimize performance in PySpark jobs? (advanced)
  • Explain the difference between map and flatMap transformations in PySpark (basic)
  • What is the role of a SparkContext in PySpark? (basic)
  • How do you handle schema inference in PySpark? (medium)
  • What is a SparkSession in PySpark? (basic)
  • How do you join DataFrames in PySpark? (medium)
  • Explain the concept of partitioning in PySpark (medium)
  • What is a UDF in PySpark? (medium)
  • How do you cache DataFrames in PySpark for optimization? (medium)
  • Explain the concept of lazy evaluation in PySpark (medium)
  • How do you handle skewed data in PySpark? (advanced)
  • What is checkpointing in PySpark and how does it help in fault tolerance? (advanced)
  • How do you tune the performance of a PySpark application? (advanced)
  • Explain the use of Accumulators in PySpark (advanced)
  • How do you handle broadcast variables in PySpark? (advanced)
  • What are the different data sources supported by PySpark? (medium)
  • How can you run PySpark on a cluster? (medium)
  • What is the purpose of the PySpark MLlib library? (medium)
  • How do you handle serialization and deserialization in PySpark? (advanced)
  • What are the best practices for deploying PySpark applications in production? (advanced)

Closing Remark

As you explore PySpark jobs in India, remember to prepare thoroughly for interviews and showcase your expertise confidently. With the right skills and knowledge, you can excel in this field and advance your career in the world of big data and analytics. Good luck!

cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Featured Companies