896 Hdfs Jobs - Page 22

Role Responsibilities: Build scalable data pipelines and applications from scratch on AWS Design and manage data ingestion processes and workflows Collaborate with functional and data science teams to deliver data solutions Optimize data storage and organization using HDFS, S3, and Delta Lake Job Requirements: Strong programming experience in Python or Java Hands-on experience with AWS services and deployment Proficiency in HDFS, S3, SQL, and data ingestion tools Exposure to Delta Lake or Databricks is an added advantage

As a PySpark Developer at our company, you will be responsible for developing and maintaining data processing pipelines using PySpark, Apache Spark's Python API. You will collaborate with data engineers, data scientists, and other stakeholders to design and implement scalable and efficient data processing solutions. Key Responsibilities: - Designing, developing, and maintaining PySpark data processing pipelines for processing structured and unstructured data - Collaborating with data engineers and data scientists to understand data requirements and design efficient data models and transformations - Optimizing and tuning PySpark jobs for performance, scalability, and reliability - Implementin...

As a Data Scientist at mPokket, you will be an integral part of a rapidly growing fintech startup in the lending domain specializing in offering microloans to college students and young professionals through a mobile app, with over 11 million loans disbursed. Backed by a US-based private equity firm, the profitable organization is poised to scale its business 10x-20x in the upcoming years. If you are an ambitious individual with a passion for working with raw data to derive meaningful insights, this is an opportunity for you to join on this incredible journey. Your primary responsibility involves collaborating with the data science team to plan and execute projects aimed at building advanced...

As a Data Engineer at our company, you will be responsible for designing, developing, and maintaining scalable data pipelines using Spark (PySpark or Spark with Scala). Your key responsibilities will include: - Constructing data ingestion and transformation frameworks for both structured and unstructured data sources. - Collaborating with data analysts, data scientists, and business stakeholders to comprehend requirements and deliver reliable data solutions. - Handling large volumes of data while ensuring quality, integrity, and consistency. - Optimizing data workflows for enhanced performance, scalability, and cost efficiency on cloud platforms such as AWS, Azure, or GCP. - Implementing dat...

Role Overview: As a Data Scientist at mPokket, you will play a crucial role in deriving valuable insights from raw data to drive business decisions and improve products. You will collaborate with the data science team, plan projects, and develop analytics models. Your problem-solving skills and expertise in statistical analysis will be key in aligning data products with business objectives. Key Responsibilities: - Oversee the data scientists team and guide colleagues on new techniques - Work closely with data and software engineers to deploy scalable technologies - Lead the conception, planning, and prioritization of data projects - Build analytic systems and predictive models - Explore new ...

SUMMARY Summary We are seeking experienced Palantir Foundry Professionals to join our team in Bangalore. The ideal candidate will have a strong background in Palantir Foundry, with a minimum of 5 years of relevant experience. This role requires a deep understanding of data integration, application development, and cloud-based deployment. The successful candidate will be responsible for troubleshooting and optimizing application performance, and will work closely with our team to deliver high-quality solutions. Responsibilities Develop and implement data integration solutions using Palantir Foundry Design and deploy cloud-based applications using Agile methodologies Troubleshoot and optimize ...

Work Location: Pan India Experience: 12+yrs Responsibility: Provide top-quality solution design and execution Provide support in defining the scope and sizing of work Align the organizations big data solutions with their Client initiatives as requested Engage with clients to understand strategic requirements Responsible for translating business requirements into technology solutions Work with domain experts to put together a delivery plan with and stay on track Utilize Big Data technologies to design, develop, and evolve scalable and fault-tolerant distributed components Organize all meetings with customers and ensure prompt resolution of gaps and roadblocks Stay current on latest technology...

Greetings from Garmin! We are hiring Senior Database Administrator (Big Data Admin). Please go through the below JD. About Company: Garmin Ltd. is an American multinational technology company founded in 1989 by Gary Burrell and Min Kao in Lenexa, Kansas, United States, with headquarters in Olathe, Kansas. Since 2010, the company is incorporated in Schaffhausen, Switzerland. The company specializes in GPS technology for automotive, aviation, marine, outdoor, and sport activities. At Garmin, we like to work hard and play hard. It comes easy when you work on cool products with hard-working individuals who share the same passion. We make products that are engineered on the inside for life on the...

As a Data Ops Capability Deployment Analyst at Citi, your role will involve applying your in-depth disciplinary knowledge to contribute to the development of new solutions and frameworks within the Enterprise Data function. You will be responsible for performing data analytics and data analysis across different asset classes, as well as building data science capabilities within the team. You will work closely with the wider Enterprise Data team to deliver business priorities and collaborate with the front to back leads. Key Responsibilities: - Hands-on experience with a data engineering background and a thorough understanding of Distributed Data platforms and Cloud services. - Sound understa...

Job Summary Candidate should have 5+ years of Banking domain project experience in Hadoop technologies. Required skill sets: Technical: Very good knowledge in Big Data Hadoop architecture, HDFS, Hive, Spark, Scala, Yarn, Zookeeper, Unix shell scripts, Control-M, DevOps. Functional: Work experience in Banking domain projects Candidate should have work experience agile projects and experience to handle multiple stakeholders and work with business users to effectively gather requirements and preparing FSD, TSD, etc. Qualifications Education-BE/ME in Computer Science Certifications-Any hadoop related Languages-English Skills And Experience Hadoop technologies: Big Data Hadoop architecture, HDFS,...

Job Description Experience in Big Data Technologies and Data Handling Experience in PySpark Knowledge & experience in SQL GCP/AWS/Azure Hadoop / HDFS/Other Databases Airflow/Other Automation tools Linux/ Bash scripting Kafka/Flume/Kinesis/Data Streaming Experience in Cloud, DevOps & CI/CD Responsibilities Experience in Big Data Technologies and Data Handling Experience in PySpark Knowledge & experience in SQL GCP/AWS/Azure Hadoop / HDFS/Other Databases Airflow/Other Automation tools Linux/ Bash scripting Kafka/Flume/Kinesis/Data Streaming Experience in Cloud, DevOps & CI/CD Qualifications B.Tech

Hiring for Product Based organization Summary: The Engineer will be responsible for designing, deploying, and automating AWS Infrastructure based on customer requirements to deliver analytical solutions. This role will work closely with Managers, Clients, and Managed Services teams. Role : Senior Platform Engineer Location : Bengaluru/Pune Total Experience: 5+ years Relevant Experience: 3+ years Responsibilities of the Role: Develop automation-first solutions for cloud data services provisioning, integrating IAM, networking, security policies as code, and observability from the ground up. Collaborate with developers and internal stakeholders to deliver automated, cloud-native, and AI-ready s...

Our client is a global commerce leader where you can influence how the world buys, sells, and gives. Youll be part of a work culture thats been genuinely committed to diversity and inclusion since its founding over twenty five years ago. Here, you can be yourself, do your best work along with a team of professionals, and have a meaningful impact on people across the globe. We seek people with drive, ideas, and a passion for helping small businesses succeed to help. About the team: We are the AI Platform Team, providing highly available, scalable, and automated machine learning and big data infrastructure for researchers and data scientists globally. We are looking for a motivated, self-relia...

To ensure efficient and effective functioning of Coating plant in shifts and proper maintenance of machines in the plant Should have Knowledge of PLC, AC / DC Drvies (GE Fanuc / siemens / schneider), Induction Unit (Electrotherm / Inductotherm), HT & LT Braker Maintenance (Schneider / L&T), AC & DC Motor Maintenance, Temperature Controller, Switchgear, Starters, SAP Knowledge Industrial Safety - Proper Housekeeping of departnental Electrical systems & spares as per OHSAS & EMS standard. Maximizing the machine uptime - Attend breakdowns in time. Analyze the breakdown & take corrective action for repetitive/non repetitive breakdowns. Documentations as per SOP (maintenance). In-house repairing ...

Role Overview: As a Strong Data Architect, Lead Data Engineer, Engineering Manager / Director in our company, you will play a crucial role in designing and implementing data engineering systems from the early stages. Your expertise in large data technologies and hands-on tech development will be instrumental in creating scalable and maintainable data architectures. Key Responsibilities: - Possess 10+ years of experience in data engineering roles, with at least 2+ years in a leadership position - Demonstrate 7+ years of hands-on tech development experience with Java (highly preferred) or Python, Node.JS, GoLang - Showcase recent 4+ years of experience with high-growth product startups, specif...

Practice Overview : Skill/Operating Group Technology Consulting Level Consultant Location Gurgaon/Mumbai/ Bangalore Travel Percentage Expected Travel could be anywhere between 0-100% Why Technology Consulting The Technology Consulting business within Capability Network invents the future for clients by providing them with the right guidance, design thinking and innovative solutions for technological transformation. As technology rapidly evolves, it's more important than ever to have an innovation advisor who can create a new vision or put one into placeto solve the client's toughest business problems. Specialize in management or technology consulting to transform the world's leading organiza...

Our client is a global commerce leader where you can influence how the world buys, sells, and gives. Youll be part of a work culture thats been genuinely committed to diversity and inclusion since its founding over twenty five years ago. Here, you can be yourself, do your best work along with a team of professionals, and have a meaningful impact on people across the globe. We seek people with drive, ideas, and a passion for helping small businesses succeed to help About the team: We are the AI Platform Team! We are looking for a highly motivated, self-reliant, experienced SRE and customer support engineer who is passionate about driving major transformation within the AI organization. This r...

Strong verbal and written communication skills Excellent analytical and problem-solving skills Hands-on and good understanding of DevOps tools and automation frameworks Demonstrated hands-on experience with DevOps techniques, building continuous integration solutions using Jenkins, Git, Maven, Nexus Experience with containerization orchestration using Docker, Docker Compose, Kubernetes Experience with configuration management tools such as Ansible, Chef, or Puppet Proficiency in at least one scripting language: Batch/PowerShell Scripting, Shell Scripting, NSIS Scripting Experience working in UNIX, Linux, and/or Windows environments Hands-on experience with cloud technology (AWS/Azure/GCP) Ex...

We are looking for an experienced Technical Lead or Technical Architect with expertise in Data Warehousing , DBT , Snowflake , Data Modeling , and Data Engineering . The ideal candidate will have strong hands-on experience in customer handling, requirements gathering, and analysis for new requirements and Change Requests (CR). You will lead and manage key technical initiatives, ensuring smooth delivery and execution of data engineering and warehousing solutions. Key Responsibilities: Lead and manage data warehousing projects, ensuring timely and efficient execution. Work extensively with Amazon Aurora , Amazon RDS , AWS DMS , Amazon DynamoDB , Oracle , and PL/SQL . Design and implement data ...

We are looking for an experienced Technical Lead or Technical Architect with expertise in Data Warehousing , DBT , Snowflake , Data Modeling , and Data Engineering . The ideal candidate will have strong hands-on experience in customer handling, requirements gathering, and analysis for new requirements and Change Requests (CR). You will lead and manage key technical initiatives, ensuring smooth delivery and execution of data engineering and warehousing solutions. Key Responsibilities: Lead and manage data warehousing projects, ensuring timely and efficient execution. Work extensively with Amazon Aurora , Amazon RDS , AWS DMS , Amazon DynamoDB , Oracle , and PL/SQL . Design and implement data ...

We are looking for an experienced Technical Lead or Technical Architect with expertise in Data Warehousing , DBT , Snowflake , Data Modeling , and Data Engineering . The ideal candidate will have strong hands-on experience in customer handling, requirements gathering, and analysis for new requirements and Change Requests (CR). You will lead and manage key technical initiatives, ensuring smooth delivery and execution of data engineering and warehousing solutions. Key Responsibilities: Lead and manage data warehousing projects, ensuring timely and efficient execution. Work extensively with Amazon Aurora , Amazon RDS , AWS DMS , Amazon DynamoDB , Oracle , and PL/SQL . Design and implement data ...

Position Summary: We are looking for candidates with hands on experience in Big Data or Cloud Technologies. Must have technical Skills 7 to 10 Years of experience Expertize and hands-on experience on Spark Data Frame, and Hadoop echo system components Must Have Good and hand-on experience* of any of the Cloud (AWS/Azure/GCP) Must Have Good knowledge of PySpark (SparkSQL) Must Have Good knowledge of Shell script & Python Good to Have Good knowledge of SQL Good to Have Good knowledge of migration projects on Hadoop Good to Have Good Knowledge of one of the Workflow engine like Oozie, Autosys Good to Have Good knowledge of Agile Development Good to Have Passionate about exploring new technologi...

As an Applications Development Technology Lead Analyst at Citigroup, your role involves establishing and implementing new or revised application systems and programs in coordination with the Technology team. Your main objective is to lead applications systems analysis and programming activities. Key Responsibilities: - Architect Data solutions for Big Data Platform - Provide solutions for business data use cases and convert them into technical implementations - Ensure application design adheres to the overall architecture blueprint - Develop standards for coding, testing, debugging, and implementation - Gain comprehensive knowledge of how different areas of business integrate to achieve busi...

As an experienced professional at Maxwell GeoSystems, you will join the team as a Senior Technical Analyst to assist in the development and implementation of Company-wide SOA Reporting. Working closely with the IT Head, your main responsibilities will include planning, runtime design, and integration of software services for effective data handling and transformation. Responsibilities: - Assist in developing and implementing Company-wide SOA for instrumentation and construction monitoring SaaS. - Oversee the planning, runtime design, and integration of software services for data handling and transformation. - Develop the logical and physical layout of the overall solution and its components....

Data Engineering & Pipeline Development Design implement and maintain ETL processes using ADF and ADB. Create and manage views in ADB and SQL for efficient data access. Optimize SQL queries for large datasets and high performance. Conduct end-to-end testing and impact analysis on data pipelines. Optimization & Performance Tuning Identify and resolve bottlenecks in data processing. Optimize SQL queries and Delta Tables for fast data processing. Data Sharing & Integration Implement Delta Share, SQL Endpoints, and other data sharing methods. Use Delta Tables for efficient data sharing and processing. API Integration & Development Integrate external systems through Databricks Notebooks and build...