265 Parquet Jobs - Page 8

You will be responsible for building and maintaining secure, scalable data pipelines using Databricks and Azure. This includes handling ingestion from various sources such as files, APIs, and streaming, performing data transformation, and ensuring quality validation. Collaboration with subsystem data science and product teams will be essential to prepare for machine learning readiness. Your technical expertise should cover working with Notebooks (SQL, Python), Delta Lake, Unity Catalog, ADLS/S3, job orchestration, APIs, structured logging, and Infrastructure as Code (IaC) tools like Terraform. Delivery practices like Trunk-based development, Test-Driven Development (TDD), Git, and Continuous...

We are seeking an experienced and detail-oriented Snowflake Developer to join our Data Engineering team. As a Snowflake Developer, you will be responsible for developing, optimizing, and maintaining data pipelines and integration processes using the Snowflake cloud data warehouse, ensuring efficient data flow across our systems. RWS Technology Services India provides end-to-end business technology solutions, offering a wide portfolio of services around digital technologies and technology operations to help organizations enhance efficiencies and reduce costs. Our team specializes in state-of-the-art technology solutions across the product lifecycle management process, from consulting and desi...

The Applications Development Sr Programmer Analyst will work in Information management team of Services Technologies, on projects focusing Big Data and public cloud adoption. It is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities. Responsibilities: Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised appli...

Join us as a Data Engineer responsible for supporting the successful delivery of Location Strategy projects to plan, budget, agreed quality and governance standards. You'll spearhead the evolution of our digital landscape, driving innovation and excellence. You will harness cutting-edge technology to revolutionise our digital offerings, ensuring unparalleled customer experiences. To be successful as a Data Engineer, you should have experience with hands-on experience in pyspark and strong knowledge of Dataframes, RDD, and SparkSQL. Additionally, experience in developing, testing, and maintaining applications on AWS Cloud is crucial. A strong hold on the AWS Data Analytics Technology Stack (G...

About the job: At Citi, we're not just building technology, we're building the future of banking. Encompassing a broad range of specialties, roles, and cultures, our teams are creating innovations used across the globe. Citi is constantly growing and progressing through our technology, with a laser focus on evolving the ways of doing things. As one of the world's most global banks, we're changing how the world does business. Shape your Career with Citi We're currently looking for a high-caliber professional to join our team as AVP- Data Engineer based in Pune, India. Being part of our team means that we'll provide you with the resources to meet your unique needs, empower you to make healthy ...

You will need to have at least 7-9 years of experience to apply for the role of Microsoft Fabric Architect. In this role, you will be responsible for working hands-on with Microsoft Fabric, which includes Lakehouse, Synapse, Pipelines, and Dataflows. You should be proficient in DAX, Power Query (M), and SQL, and have a strong understanding of Delta Lake, Parquet, and OneLake storage. Knowledge of Spark Notebooks, KQL, and Data Engineering within Fabric will be an added advantage. Experience with source control integration (Git) and CI/CD pipelines is essential for this role. Familiarity with Azure Data Factory, Azure Synapse Analytics, or Databricks will be considered a plus. You should poss...

We have a great opportunity for the role of Senior Data Engineer - Azure, Databricks. Relevant Exp: 5+ Yrs Job Description:- Mandatory Skills -Platform Engineer, Databricks, Azure Cloud, Data Engineer, Pyspark, SQL, Databricks. o Hands-on in data structures, distributed systems, Spark, SQL, PySpark, NoSQL Databases. o Strong software development skills in at least one of: Python, Pyspark or Scala. o Develop and maintain scalable & modular data pipelines using databricks & Apache Spark. o Experience building and supporting large-scale systems in a production environment. o Experience working on either of the cloud storages - AWS S3, Azure data lake, GCS buckets. o Exposure to the ELT tools of...

Join our transformation within the RMG Data Engineering team in Hyderabad and you will have the opportunity to work with a collaborative and dynamic network of technologists. Our teams play a pivotal role in implementing data products, creating impactful visualizations, and delivering seamless data solutions to downstream systems. At Macquarie, our advantage is bringing together diverse people and empowering them to shape all kinds of possibilities. We are a global financial services group operating in 31 markets and with 56 years of unbroken profitability. Youll be part of a friendly and supportive team where everyone - no matter what role - contributes ideas and drives outcomes. What role ...

At Iron Mountain, we believe that work, when done well, can have a positive impact on our customers, employees, and the planet. That's why we are looking for smart and committed individuals to join our team. Whether you are starting your career or seeking a change, we invite you to explore how you can enhance the impact of your work at Iron Mountain. We offer expert and sustainable solutions in records and information management, digital transformation services, data centers, asset lifecycle management, and fine art storage, handling, and logistics. Collaborating with over 225,000 customers worldwide, we aim to preserve valuable artifacts, optimize inventory, and safeguard data privacy throu...

As a Data Engineering Senior Specialist (Databricks) at Nasdaq Bangalore, you will be joining the Bangalore technology center in India, where innovation and effectiveness are the driving forces. Nasdaq is at the forefront of revolutionizing markets and constantly evolving by adopting new technologies to create innovative solutions, aiming to shape the future. In this role, your primary responsibility will be to analyze defined business requirements, providing analytical insights, modeling, dimensional modeling, and testing to design solutions that meet customer needs effectively. You will focus on understanding business data needs and translating them into adaptable, extensible, and sustaina...

Key Responsibilities: Assist a team of engineers to provide production support and troubleshooting for 24/7 manufacturing operations. Solve customer problems by analyzing situations to find common issues, identifying root causes, and developing effective solutions. Document applications and solutions. Respond to customer requests. Participate in project teams and represent interests in area(s) of responsibility (e.g., screening, measurements, warehouse, commercial systems, etc.). Lead local IT projects. Arrange/conduct quality assurance testing on technology. Ensure that confidentiality and protection of information is built into applications and processes. Respond to customers and IT leader...

As a Data Modeler, your primary responsibility will be to design and develop conceptual, logical, and physical data models supporting enterprise data initiatives. You will work with modern storage formats like Parquet and ORC, and build and optimize data models within Databricks Unity Catalog. Collaborating with data engineers, architects, analysts, and stakeholders, you will ensure alignment with ingestion pipelines and business goals. Translating business and reporting requirements into robust data architecture, you will follow best practices in data warehousing and Lakehouse design. Your role will involve maintaining metadata artifacts, enforcing data governance, quality, and security pro...

???? Were Hiring: Senior Data Engineer 7+ Years Experience ???? Location: Gurugram, Haryana, India ???? Duration: 6 Months C2H (Contract to Hire) ???? Apply Now: [HIDDEN TEXT] ???? What Were Looking For: ? 7+ years of experience in data engineering ? Strong expertise in building scalable, robust batch and real-time data pipelines ? Proficiency in AWS Data Services (S3, Glue, Athena, EMR, Kinesis, etc.) ? Advanced SQL skills and deep knowledge of file formats: Parquet, Delta Lake, Iceberg, Hudi ? Hands-on experience with CDC patterns ? Experience with stream processing (Apache Flink , Kafka Streams) and distributed frameworks like PySpark ? Expertise in Apache Airflow for workflow orchestrati...

About us -Coders Brain is a global leader in its services, digital and business solutions that partners with its clients to simplify, strengthen and transform their businesses. We ensure the highest levels of certainty and satisfaction through a deep-set commitment to our clients, comprehensive industry expertise and a global network of innovation and delivery centers. We achieved our success because of how successfully we integrate with our clients. Quick Implementation - We offer quick implementation for the new onboarding client. Experienced Team - We've built an elite and diverse team that brings its unique blend of talent, expertise, and experience to make you more successful, ensuring ...

You should have familiarity with modern storage formats like Parquet and ORC. Your responsibilities will include designing and developing conceptual, logical, and physical data models to support enterprise data initiatives. You will build, maintain, and optimize data models within Databricks Unity Catalog, developing efficient data structures using Delta Lake to optimize performance, scalability, and reusability. Collaboration with data engineers, architects, analysts, and stakeholders is essential to ensure data model alignment with ingestion pipelines and business goals. You will translate business and reporting requirements into a robust data architecture using best practices in data ware...

Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education. As an ETL Technical Lead at Orion Innovation located in Chennai, you are required to have at least 5 years of ETL experience and 3 ...

Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Palantir Foundry Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Project Role:Lead Data Engineer Project Role Description:Design, build and enhance applications to meet business process and requirements in Palantir foundry.Work experience:Minimum 6 years Must have Skills: Palantir Foundry, PySparkGood to Have Skills: Experience in PySpark, python and SQLKnowledge on Big Data tools & TechnologiesOrganizational and project management experien...

As an Azure Data Engineer within our team, you will play a crucial role in enhancing and supporting existing Data & Analytics solutions by utilizing Azure Data Engineering technologies. Your primary focus will be on developing, maintaining, and deploying IT products and solutions that cater to various business users, with a strong emphasis on performance, scalability, and reliability. Your responsibilities will include incident classification and prioritization, log analysis, coordination with SMEs, escalation of complex issues, root cause analysis, stakeholder communication, code reviews, bug fixing, enhancements, and performance tuning. You will design, develop, and support data pipelines ...

You are a Java Developer with AI/ML experience, required to have at least 5+ years of industry experience in Java, Spring Boot, Spring Data, and a minimum of 2 years of AI/ML project or professional experience. You should possess a strong background in developing and consuming REST APIs and asynchronous messaging using technologies like Kafka or RabbitMQ. Your role involves integrating AI/ML models into Java services or making calls to external ML endpoints. You need to have a comprehensive understanding of the ML lifecycle encompassing training, validation, inference, monitoring, and retraining. Familiarity with tools such as TensorFlow, PyTorch, Scikit-Learn, or ONNX is essential. Previous...

We are looking for a skilled Data Engineer to join our team, working on end-to-end data engineering and data science use cases. The ideal candidate will have strong expertise in Python or Scala, Spark (Databricks), and SQL, building scalable and efficient data pipelines on Azure. Responsibilities include designing, building, and maintaining scalable ETL/ELT data pipelines using Azure Data Factory, Databricks, and Spark. Developing and optimizing data workflows using SQL and Python or Scala for large-scale data processing and transformation. Implementing performance tuning and optimization strategies for data pipelines and Spark jobs to ensure efficient data handling. Collaborating with data ...

Position level: AI Specialist/Software engineering professional (Cant be a junior/fresher, needs to be middle to senior level person) Work Experience: Ideally, 4 to 5 years of working as a Data Scientist / Machine Learning and AI at a managerial position (end-to-end project responsibility). Slightly lower work experience can be considered based on the skill level of the candidate. About the job: Use AI-ML to work with data to predict process behaviors. Stay abreast of industry trends, emerging technologies, and best practices in data science, and provide recommendations for adopting innovative approaches within the product teams. In addition, championing a data-driven culture, promoting best...

Tiger Analytics is a global AI and analytics consulting firm with a team of over 2800 professionals focused on using data and technology to solve complex problems that impact millions of lives worldwide. Our culture is centered around expertise, respect, and a team-first mindset. Headquartered in Silicon Valley, we have delivery centers globally and offices in various cities across India, the US, UK, Canada, and Singapore, along with a significant remote workforce. At Tiger Analytics, we are certified as a Great Place to Work. Joining our team means being at the forefront of the AI revolution, working with innovative teams that push boundaries and create inspiring solutions. We are currently...

About Calfus: Calfus is a Silicon Valley headquartered software engineering and platforms company with a vision deeply rooted in the Olympic motto "Citius, Altius, Fortius Communiter". At Calfus, we aim to inspire our team to rise faster, higher, and stronger while fostering a collaborative environment to build software at speed and scale. Our primary focus is on creating engineered digital solutions that drive positive impact on business outcomes. Upholding principles of #Equity and #Diversity, we strive to create a diverse ecosystem that extends to the broader society. Join us at #Calfus and embark on an extraordinary journey with us! Position Overview: As a Data Engineer specializing in B...

About the Role We are seeking a Senior Python Developer to lead the development of scalable, high-performance backend systems and data-driven applications. You will work on building robust services, APIs, and data workflows that power real-time and batch applications. This role is ideal for a seasoned developer who thrives on solving complex problems, building reliable software, and working across data and engineering teams. Youll play a key role in designing and developing Python-based systems that interface with big data frameworks, cloud platforms, and analytics tools. Key Responsibilities Design and develop robust Python-based backend services and microservices. Build RESTful APIs and in...

Role Description Role Description: Senior Scala Data Engineer Scala Data Engineer needs to be able to understand existing code and help refactor, and migrate into new environment. Role and responsibilities * Read existing scala spark code. * Create unit tests for scala spark code * Enhance and Write scala spark code. * Proficient in working with S3 file with csv and parquet format. * Proficient in working with mongodb. Building up environments independently to test assigned work, Execute manual and automated tests. Experience with enterprise tools, like Git, Azure, TFS. Experience with JIRA or similar defect tracking tool. Engage and participate on an Agile team of a world-class software dev...