263 Parquet Jobs

Hiring: Data Analyst Location: Pan India (Hybrid) Experience: 6+ Years Notice Period: Immediate 30 Day Job Description We are seeking an experienced Data Analyst with a strong background in Databricks, SQL, and the Banking domain . The ideal candidate will have hands-on experience in developing and managing data pipelines , performing data analysis , and delivering insights to support business decisions. Key Responsibilities Develop, manage, and optimize data pipelines and data infrastructure within Azure. Perform data analysis to support reporting, MIS, and decision-making functions. Work on end-to-end Data Product deployment , from design to production. Collaborate with cross-functional te...

Meet Our Team: The Software Engineer works with a team of developers of varied skillsets (together known as Caterpillars scrum team) to design and develop the best of breed software offerings for the CRM industry. The D&AI team combine their industry knowledge and Pegas innovative technology to create products that will be used to solve business problems or inefficiencies for our customers. We are seeking an experienced technical professional who can demonstrate creativity, teamwork, a results-driven attitude, and a dedication to customer success. Picture Yourself at Pega: Role Overview We are seeking a highly skilled and motivated Software Engineer to join our Machine Learning & AI team. Th...

SUMMARY Key Responsibilities - Debug and resolve data ingestion and mapping issues for clients. - Understand current data mappings for file ingestions (FinTech files) for each client. - Work with ETL/ELT pipelines and messaging systems (Kafka or any custom ingestion services). - Validate and process data in common formats (CSV, JSON, Parquet). - Monitor and optimize data pipelines with Prometheus/Grafana alerts. - Collaborate with software engineers on schema evolution and data contracts. - Contribute automation scripts in Python/Shell to reduce repetitive tasks. - Participate in on-call for production data issues. RequirementsRequired Skills - Strong knowledge of SQL and PostgreSQL. - Exper...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing...

We are hiring a Data Engineer with strong experience in AWS cloud services, Python, and data pipeline development. The ideal candidate will work on designing scalable data solutions, building ETL pipelines, and collaborating with cross-functional teams to deliver high-quality data products. Key Responsibilities: Design and develop data pipelines using AWS Glue, Lambda, Airflow, and S3. Build dimensional data models and dashboards using Power BI. Implement CI/CD pipelines for automated deployments. Work with APIs, JSON/XML data formats, and data governance practices. Collaborate with data scientists and business teams to deliver insights.

Key Responsibilities Architect and Develop Data Pipelines: Design and implement data pipelines that can handle the 4Vs of data (Volume, Variety, Velocity, Veracity) from diverse sources. Implement CDC: Utilize Change Data Capture (CDC) to efficiently and reliably synchronize data between source and target systems. Build a Medallion Architecture: Design and implement a data platform following the Medallion Architecture (Raw, Refined, Insights) to ensure data quality, manage transformations, and support analytical needs. Orchestrate and Manage Workflows: Develop and manage data orchestration processes to automate data ingestion, transformation, and delivery. Data Strategy & Business Value: Art...

You have a strong background in data engineering or a related field with over 8 years of experience. Your expertise includes hands-on experience with Azure Databricks, Spark, Python/Scala, CICD, and scripting for data processing. You have worked with multiple file formats such as Parquet, Delta, and Iceberg, and have knowledge of streaming technologies like Kafka for real-time data ingestion. Additionally, you are experienced in data governance and data security in Azure, and have a deep understanding of Azure Data Services. Key Responsibilities: - Utilize Azure Databricks, Spark, Python, and Scala for scalable and distributed data processing. - Work with various file formats like Parquet, D...

At JPMorgan Chase, you'll have the opportunity to advance your career and push the boundaries of data engineering by developing innovative, secure solutions that transform financial services. As a Data Engineer at JPMorgan Chase within the Employee Platforms team, you will become an integral part of an agile team dedicated to enhancing, building, and delivering trusted market-leading technology products in a secure, stable, and scalable way. Job Responsibilities Execute creative software and data solutions, including design, development, and technical troubleshooting, by thinking beyond routine or conventional approaches to build solutions or break down technical problems. Develop secure, hi...

About Us At SentinelOne, we're redefining cybersecurity by pushing the limits of what's possibleleveraging AI-powered, data-driven innovation to stay ahead of tomorrow's threats. From building industry-leading products to cultivating an exceptional company culture, our core values guide everything we do. We're looking for passionate individuals who thrive in collaborative environments and are eager to drive impact. If you're excited about solving complex challenges in bold, innovative ways, we'd love to connect with you. What Are We Looking For SentinelOne is seeking a Senior Software Engineer to join the Observo AI team in India, our cutting-edge AI-driven data pipeline optimization platfor...

Key Responsibilities Architect and Develop Data Pipelines: Design and implement data pipelines that can handle the 4Vs of data (Volume, Variety, Velocity, Veracity) from diverse sources. Implement CDC: Utilize Change Data Capture (CDC) to efficiently and reliably synchronize data between source and target systems. Build a Medallion Architecture: Design and implement a data platform following the Medallion Architecture (Raw, Refined, Insights) to ensure data quality, manage transformations, and support analytical needs. Orchestrate and Manage Workflows: Develop and manage data orchestration processes to automate data ingestion, transformation, and delivery. Data Strategy & Business Value: Art...

Computer science or similar education background. 7+ years total work experience. 5+ years hands on development experience in big data (Hadoop (cloudera preferred), spark, hive, pyspark, beeline, Scala, YARN, spark-submit, spark-java, parquet, ranger). Good understanding of big data storage concepts like partitioning, query and storage optimization, backup, restore and data replication. 5+ years experience in SDLC models (waterfall, and agile). 2+ years java coding experience. 2+ years experience in unix scripting and RDBMS Sql (e.g. sql server) Nice-to-have kills : Capital markets work experience Reference data (securities, trading books, client data, etc) Other systems involving large data...

Shift: 2:00 PM 11:00 PM IST Experience: 8+ years of hands-on Data Engineering experience About The Role We are looking for experienced Data Engineers who can design, build, and optimize large-scale data pipelines. This role is for individual contributors who love coding, problem-solving, and working with cutting-edge big data technologies. Key Responsibilities Design, develop, and maintain scalable data pipelines for ETL processes using Spark (PySpark/Scala). Optimize data flow and automate existing processes for improved performance. Collaborate with cross-functional teams Data Science, Analytics, and Product to support data infrastructure needs. Work on data quality, reliability, and perfo...

As a Data Engineer, your role will involve collaborating with the Lead Data Engineer and Data Architect to design and implement end-to-end data solutions. You will create, test, and implement enterprise-level apps with Snowflake, as well as design and implement features for identity and access management. Your responsibilities will also include creating authorization frameworks for better access control and implementing novel query optimization and major security competencies with encryption. Additionally, you will be solving performance issues and scalability issues in the system, managing transaction management with distributed data processing algorithms, and possessing ownership right fro...

Job Title Data Engg - Snowflake Role Summary & Role Description We are seeking a talented Senior Software Engineer to join our agile team. The successful candidate will play a crucial role in ensuring the quality and reliability of our software products, with a focus on developing Azure-based applications, data warehouse and ETL backend systems. Proficiency in Data Warehousing and Snowflake is essential for this role. Candidate must have minimum of 8 years of relevant experience in Data Warehousing and Database application development. Candidate must have hands on experience in Snowflake and Azure. Analyze, Design and Code snowflake objects and views. Create Database Views/materialized views...

Job Title Data Engg - Snowflake Role Summary & Role Description We are seeking a talented Senior Software Engineer to join our agile team. The successful candidate will play a crucial role in ensuring the quality and reliability of our software products, with a focus on developing Azure-based applications, data warehouse and ETL backend systems. Proficiency in Data Warehousing and Snowflake is essential for this role. Candidate must have minimum of 8 years of relevant experience in Data Warehousing and Database application development. Candidate must have hands on experience in Snowflake and Azure. Analyze, Design and Code snowflake objects and views. Create Database Views/materialized views...

Location : Bangalore, Chennai, Pune & Hyderabad Must have skills: ETL/ SQL with pyspark Experience and expertise in pipeline/ETL (Extract, Transform, Load) processes, API integration, scripting languages (Python), and big data technologies (Trio, Iceberg, DuckDB/Parquet). Database design, data modeling, and data warehousing. SQL and at least one cloud platform. Analytical tools (Superset, Power BI), statistical analysis, and SQL. To derive insights from data. Data governance principles and data quality. Data management, data integration projects using IPAAS tools like IBM Sterling, Mulesoft, Dell Boomi About Finacle Finacle is an industry leader in digital banking solutions. We partner with ...

where youll be supported and inspired by a collaborative community of colleagues around the world, where youll be supported and inspired bya collaborative community of colleagues around the world, and where youll be able to reimagine whats possible. Join us and help the worlds leading organizationsunlock the value of technology and build a more sustainable, more inclusive world. Your Role As a senior software engineer with Capgemini, you will have 3 + years of experience in Scala with strong project track record. Hands-on expertise in Scala and Spark, strong SQL skills on DB2, and proficiency in handling diverse file formats including JSON, Parquet, AVRO, ORC, and XML. Proven experience in H...

As an AWS Glue Developer with 6 to 9 years of experience, you will be responsible for the following: - Hands-on experience in AWS Glue ETL development for at least 3 years - Strong programming skills in Python and PySpark for data transformation - Proficiency in AWS services such as S3, Redshift, Athena, Lambda, and EMR - Experience working with data formats like Parquet, Avro, JSON, CSV, and ORC - Advanced SQL skills for data querying and transformation - Deep understanding of ETL/ELT design patterns and best practices - Knowledge of dimensional modeling, star/snowflake schemas - Experience with Git/Bitbucket for code management Preferred skills for this role include: - Experience with Vang...

Role Overview: You are an experienced Azure Databricks + SQL Developer / Big Data Engineer responsible for designing, developing, and maintaining scalable data solutions on Azure. Your primary focus will be on building efficient ETL/ELT pipelines, optimizing SQL queries, and leveraging Databricks and other Azure services for advanced data processing, analytics, and data platform engineering. Your strong background in traditional SQL development and modern big data technologies on Azure will be crucial for this role. Key Responsibilities: - Develop, maintain, and optimize ETL/ELT pipelines using Azure Databricks (PySpark/Spark SQL). - Write and optimize complex SQL queries, stored procedures,...

You will thoroughly analyze the existing MATLAB codebase, algorithms, data structures, and functionalities to gain a comprehensive understanding of their purpose, implementation details, and underlying mathematical/scientific principles. You will document the intricacies of the MATLAB tools, identifying core logic, dependencies, and potential challenges for migration. Collaborate with end-users and domain experts of the current MATLAB tools to capture implicit requirements, usage patterns, and critical performance characteristics. Develop detailed migration plans, including phased approaches, risk assessments, and resource estimates for converting MATLAB components to Python. Design the arch...

We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible. As a Lead Software Engineer at JPMorgan Chase within the Corporate & Investment Bank Technology team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm's business objectives Job responsibilities Executes creative software solutions, design, development, and technical troublesh...

About the company Our client is a global, AI-powered EdFinTech company, dedicated to simplifying and democratizing access to education financing for students pursuing studies abroad. By leveraging technology and strategic partnerships, our client aims to provide seamless, transparent, and affordable financial solutions to empower the next generation of global leaders. Responsibilities The ideal candidate will bring strong experience in building data pipelines, managing data infrastructure, and deploying ML models into production. This is a hands on, high-impact role that blends data engineering and machine learning to drive actionable intelligence across the business. Responsibilities: Data ...

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards. WHY JOIN US If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you! ABOUT THE ROLE As a Lead Data Engineer (Python) , you'll drive the development and optimization of data workflows, ensuring scalable, efficient, and high-quality solutions across Python and Spark environments. You'll take end-to-end ownership of features, collaborate with global s...

Company Description SuprSend is a central communication stack based in Bengaluru, providing a centralized notification infrastructure for easily creating, managing, and delivering notifications to end users on multiple channels. The platform ensures reliable and scalable notification delivery, eliminating the need for in-house development of notification services for transactional/engagement notifications. Qualifications Computer Science and Software Development skills Bachelor's or Master's degree in Computer Science or related field Strong problem-solving and analytical skills Ability to work effectively in a team environment Experience with Go programming language is a must Key Responsibi...

GCP/Bigdata Provides technical leadership in Big Data space (Hadoop Stack like Spark, M/R, HDFS, Hive, HBase etc) and contributes to open source Big Data technologies. Must have : Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer servics) Should be aware with columnar database e.g parquet, ORC etc Visualize and evangelize next generation infrastructure in Cloud platform/Big Data space (Batch, Near Real-time, Real-time technologies). Passionate for continuous learning, experimenting, applying and contributing towards cutting edge open source technologies and software paradigms Developin...