575 Hdfs Jobs - Page 4

Required Experience: 8+ years as a hands-on Solutions Architect and/or Data Engineer designing and implementing data solutions Team lead, and/or mentorship of other engineers Ability to develop end-to-end technical solutions into production - and to help ensure performance, security, scalability, and robust data integration. Programming expertise in Java, Python and/or Scala Core cloud data platforms including Snowflake, AWS, Azure, Databricks and GCP SQL and the ability to write, debug, and optimize SQL queries Client-facing written and verbal communication skills and experience Create and deliver detailed presentations Detailed solution documentation (e.g. including POCS and roadmaps, sequ...

Job Description: 5 years of in depth hands on experience of developing testing deployment and debugging of Spark Jobs using Scala in Hadoop Platform In depth knowledge of Spark Core working with RDDs Spark SQL In depth knowledge on Spark Optimization Techniques and Best practices Good Knowledge of Scala Functional Programming Try Option Future Collections Good Knowledge of Scala OOPS Classes Traits and Objects Singleton and Companion Case Classes Good Understanding of Scala Language Features Type System Implicit Givens Hands on experience of working in Hadoop Environment HDFS Hive AWS S3 EMR Working experience on Workflow Orchestration tools like Airflow Oozie Working with API calls in Scala...

Greetings from TCS !!! TCS presents an excellent opportunity for Pyspark Developers Experience (Yrs) : 6-12 years Location: Chennai/Bangalore Mode of interview : Virtual Key Responsibilities: Develop, optimize, and maintain big data pipelines using PySpark on distributed computing platforms. Design and implement ETL workflows for ingesting, processing, and transforming large datasets in Hive . Work with structured and unstructured data sources to ensure efficient data storage and retrieval. Optimize Hive queries and Spark jobs for performance, scalability, and cost efficiency. Implement best practices for data engineering, including data governance, security, and compliance. Monitor, trouble...

Collaborate with software engineers, business stake holders and/or domain experts to translate business requirements into product features, tools, projects. ? Develop, implement, and deploy ETL solutions. ? Preprocess and analyze large datasets to identify patterns, trends, and insights. ? Evaluate, validate, and optimize data models to ensure efficiency, and generalizability. ? Monitor and maintain the performance of data pipeline, data models in production environments, identifying opportunities for improvement and update as needed. ? Document development processes, results, and lessons learned to facilitate knowledge sharing and continuous improvement.

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Principal Apache Hadoop Engineer at BairesDev We are seeking a Principal Apache Hadoop Engineer with deep expertise in big data ecosystem, HDFS ...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Senior Apache Hadoop Developer at BairesDev We are seeking a Senior Apache Hadoop Developer with expertise in big data ecosystem, HDFS architect...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Principal Apache Ozone Engineer at BairesDev We are seeking a Principal Apache Ozone Engineer with deep expertise in distributed storage systems...

4 Years Data Engineering Experience 5 years of consulting experience Completed Data Engineering Professional certification & required classes Minimum 2-3 projects delivered with hands-on experience in Databricks Completed Apache Spark Programming with Databricks, Data Engineering with Databricks, Optimizing Apache Spark on Databricks Experience in Spark and or Hadoop, Flink, Presto, other popular big data engines Familiarity with Databricks multi-hop pipeline architecture

2 Years Data Engineering Experience, 3 years of consulting experience Completed Data Engineering Associate certification & required classes Minimum 1 project delivered with hands-on experience in development on Databricks Completed Apache Spark Programming with Databricks, Data Engineering with Databricks, Optimizing Apache Spark on Databricks SQL delivery experience, and familiarity with Bigquery, Synapse or Redshift Proficient in Python, knowledge of additional databricks programming languages (Scala)

As a Streamsets ETL Developer Associate at Deutsche Bank in Pune, India, you will play a crucial role in Project Proteus, focused on transforming DWS into a standalone Asset Management platform. Your responsibilities will involve developing, testing, and maintaining robust ETL processes to facilitate data integration initiatives. **Key Responsibilities:** - Create high-quality software following standard coding practices. - Engage in hands-on code development and comprehensive testing of ETL solutions/pipelines. - Perform code reviews for team members and take ownership of end-to-end project delivery. - Translate business requirements into technical designs and collaborate effectively across...

Job Description JOB DESCRIPTION Software Engineer AMD Finance Data Engineering Engineers in the AMD Finance Data Engineering help ensure AMD division meets all of its financial control and reporting obligations. Working in small and nimble project teams, they build critical and complex applications to calculate Financial Metrics and monitor the division's Assets, Revenues and Expenses and support various financial and management reporting needs across multiple business teams. Using a combination of computer science, finance and math skills, they build and support complex and critical software products that meet the ever-changing business needs and regulatory environment. They collaborate ver...

Responsibilities : 5+ years of in depth hands on experience of developing, testing, deployment and debugging of Spark Jobs using Scala in Hadoop Platform In depth knowledge of Spark Core, working with RDDs, Spark SQL In depth knowledge on Spark Optimization Techniques and Best practices Good Knowledge of Scala Functional Programming: Try, Option, Future, Collections Good Knowledge of Scala OOPS: Classes, Traits and Objects (Singleton and Companion), Case Classes? Good Understanding of Scala Language Features: Type System, Implicit/Givens Hands on experience of working in Hadoop Environment (HDFS/Hive), AWS S3, EMR Working experience on Workflow Orchestration tools like Airflow, Oozie Working...

Position Title: Senior Software Engineer The Group: Morningstar's Research group provides independent analysis on individual securities, funds, markets, and portfolios. The Research group also provides data on hundreds of thousands of investment offerings, including stocks, mutual funds, and similar vehicles, along with real-time global market data on millions of equities, indexes, futures, options, commodities, and precious metals, in addition to foreign exchange and Treasury markets. Morningstar is one of the largest independent sources of fund, equity, and credit data and research in the world, and our advocacy for investors interests is the foundation of our company. The Role: As a Senio...

As a Talend ETL Lead, you will be responsible for leading the design and development of scalable ETL pipelines using Talend, integrating with big data platforms, and mentoring junior developers. This is a high-impact, client-facing role requiring hands-on leadership and solution ownership. Key Responsibilities: - Lead the end-to-end development of ETL pipelines using Talend Data Fabric. - Collaborate with data architects and business stakeholders to understand requirements. - Build and optimize data ingestion, transformation, and loading processes. - Ensure high performance, scalability, and reliability of data solutions. - Mentor and guide junior developers in the team. - Troubleshoot and r...

As a Data Storage Architect at our company, your role involves leading the data storage architecture and designing systems for big data. You will work closely with the development and technical teams, developing backup, restore, and monitoring solutions for our data storage infrastructure. Additionally, you will mentor the technical team on data management and big data, working in an agile mode to contribute to building a platform for streaming real-time industrial data. Key Responsibilities: - Lead the data storage architecture - Design and develop systems for big data - Work closely with the development and technical teams - Develop backup, restore, and monitoring solutions for our data st...

As a Senior Machine Learning Engineer at our company, your role will involve designing, developing, testing, deploying, and maintaining scalable, secure software for batch and real-time ML pipelines. You will optimize pipelines to be distributed, parallel, and/or multithreaded and collaborate with cross-functional teams to troubleshoot and optimize data pipelines. Demonstrating subject matter expertise and ownership for your team's services will be essential. Additionally, you will present results to stakeholders and executives, collaborate with Data Science, Big Data, and DevOps teams, and elevate team performance through mentoring, training, code reviews, and unit testing. You will also ta...

As a Senior Product Engineer located in Bengaluru, your primary responsibility will be product engineering, design, research, and development. You will utilize analytical skills and excellent communication to ensure smooth workflows and successful project outcomes. Key Responsibilities: - Help build a quality product adhering to best practices and coding standards as a full-stack engineer - Work on both the back-end and front-end of systems, developing fully-fledged platforms - Play a key role in the development of the core product, working directly with the Senior Engineers and Product team - Understand, evaluate, and deliver quality work products based on requirements - Be responsible and ...

Join us as a Developer at Barclays, where you'll take part in the evolution of our digital landscape, driving innovation and excellence. You'll harness cutting-edge technology to revolutionize our digital offerings, ensuring unparalleled customer experiences. As a part of the team, you will deliver technology stack, using strong analytical and problem solving skills to understand the business requirements and deliver quality solutions. You'll be working on complex technical problems that will involve detailed analytical skills and analysis. This will be done in conjunction with fellow engineers, business analysts and business stakeholders. To be successful as a Developer you should have expe...

As the Data Practice Head at our company, you will play a pivotal role in shaping and implementing our data strategy, fostering innovation, and establishing a high-performing data practice. Your role will require you to possess visionary leadership qualities along with a profound understanding of data management, analytics, and AI/ML technologies, complemented by a robust business acumen to align data endeavors with organizational objectives. It is imperative that you demonstrate proficiency in designing, developing, and delivering data solutions across the board. Key Responsibilities: - Define and execute the overarching strategy for the Data Practice in alignment with business goals. - Dri...

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Senior Apache Ozone Developer at BairesDev We are seeking a Senior Apache Ozone Developer with expertise in distributed storage systems, object ...

Position Title: Senior Software Engineer The Group: Morningstar’s Research group provides independent analysis on individual securities, funds, markets, and portfolios. The Research group also provides data on hundreds of thousands of investment offerings, including stocks, mutual funds, and similar vehicles, along with real-time global market data on millions of equities, indexes, futures, options, commodities, and precious metals, in addition to foreign exchange and Treasury markets. Morningstar is one of the largest independent sources of fund, equity, and credit data and research in the world, and our advocacy for investors’ interests is the foundation of our company. The Role: As a Seni...

About Happiest Minds:** Happiest Minds is a leading digital transformation and technology services company, empowered by the mission of enhancing the happiness of our customers, employees, and society at large. We provide cutting-edge solutions leveraging advanced technologies to drive meaningful outcomes for our clients. Location: Manyata Tech Park. JD: Need someone with GCP + Looker / LookML experience Strong knowledge of data processing in GCP , Spark Deep understanding of Big Query architecture, best practices, and performance optimization. Experience with DataProc for running Hadoop/ Spark jobs on GCP. Proficiency in LookML for building data models and metrics. Knowledge of building rep...

Location Gurgaon Experience 5-10 years Role Big Data Architect. About the Role: As a Big Data Engineer, you will play a critical role in integrating multiple data sources, designing scalable data workflows, and collaborating with data architects, scientists, and analysts to develop innovative solutions. You will work with rapidly evolving technologies to achieve strategic business goals. Must-Have Skills: 4+ year's of mandatory experience with Big data. 4+ year's mandatory experience in Apache Spark. Proficiency in Apache Spark, Hive on Tez, and Hadoop ecosystem components. Strong coding skills in Python & Pyspark. Experience building reusable components or frameworks using Spark Expertise i...

Let's be unstoppable together! At Circana, we are fueled by our passion for continuous learning and growth, we seek and share feedback freely, and we celebrate victories both big and small in an environment that is flexible and accommodating to our work and personal lives. We're a global company dedicated to fostering inclusivity and belonging. We value and celebrate the unique experiences, cultures, and viewpoints that each individual brings. By embracing a wide range of backgrounds, skills, expertise, and beyond, we create a stronger, more innovative environment for our employees, clients, and communities. Circana is proud to be Certified by Great Place To Work. This prestigious award is b...

Role Overview: You will be a part of the Maps team in Hyderabad, working in a dynamic environment that emphasizes flexibility, work-life balance, continuous learning, and leadership mentoring. The team follows a hybrid work policy, giving you the flexibility to perform at your best. Fun is also a priority, with opportunities for games, events, and outings to ensure an engaging work environment. Key Responsibilities: - Analyze, visualize, process, manage, and curate data on a large scale - Combine disparate signals such as data analytics, community engagement, and user feedback to enhance the Maps Platform - Design and implement complex algorithms for resolving map errors - Review solutions w...