35346 Spark Jobs - Page 35

Technical Skills Required: ETL Concepts: Strong understanding of Extract, Transform, Load (ETL) processes. Ability to design, develop, and maintain robust ETL pipelines. Database Fundamentals: Proficiency in working with relational databases (e.g., MySQL, PostgreSQL, Oracle, or MS SQL Server). Knowledge of database design and optimization techniques. Basic Data Visualization: Ability to create simple dashboards or reports using visualization tools (e.g., Tableau, Power BI, or similar). Query Optimization: Expertise in writing efficient, optimized queries to handle large datasets. Testing and Documentation: Experience in validating data accuracy and integrity through rigorous testing. Ability...

Job Overview: We are seeking an experienced Senior Developer with strong expertise in Python, Node.js, Azure, and a proven track record of migrating cloud applications from AWS to Azure. This role requires hands-on experience in PySpark, Databricks, and Azure data services such as ADF and Synapse Spark. The ideal candidate will lead end-to-end modernization and migration initiatives, code remediation, and deployment of serverless and microservices-based applications in Azure. Key Responsibilities: Lead the migration of Python and Node.js applications from AWS to Azure. Analyze legacy AWS architecture, source code, and cloud service dependencies to identify and implement code refactoring and ...

Role & responsibilities Job Title Big Data Developer Location BLR hybrid Interview rounds – 2-3 max Contract length – 1 year Tip – SQL & Scala/Spark combination mandatory. Snowflake DBT good to have. Job description: 3 years of hands-on Spark SQL, Scala, Python programming Strong knowledge and at least 2 years experience of SQL (Hive, Postgres, HWC tables) Candidates must have strong verbal, written and communication skills Good to have skillset – DBT, Snowflake Development experience Experience with on-premises environments – Cloudera and Cloud AWS and Azure. Experience with Airflow. Experience with Clinical data Agile and Lean SDLC process Experience with Java and batch scripts Key Respons...

Fully Remote Salary: $20,000-$30,000 Annually (₹17,74,950 – ₹26,62,425 INR) Key Responsibilities Design, develop, and deploy machine learning models and pipelines. Work with large, complex datasets to extract actionable insights. Collaborate with data scientists to transition research models into production-ready systems. Implement feature engineering, model training, testing, and performance optimization. Develop and maintain APIs or services to integrate ML models into applications. Evaluate model performance and conduct A/B testing or model retraining. Utilize cloud services (AWS, GCP, Azure) for scalable ML deployment. Stay current with advancements in ML frameworks and techniques. Requi...

Warning - We are a super lean and young team of 100 -+ If growth is what you aspire then we should talk. If you are looking for a 9-6 role, this is NOT for you! We are not glorifying long hours but at this juncture we need HUSTLERS who have a fire to grow and a positive intent. We have your BACK if you have OURS! Please read the full JD As one of our core team members, you'll be helping shape our sales & marketing culture at Skillinabox for Fashion Designing & Make-up Artistry. Working closely with the Founders and the Board, you will be directly responsible to lead our GTM strategies, build a customer base & work with campaigns. A "Beacon Role" to shape our growth in all dimensions and buil...

Abacus Insights is a mission-driven, rapid growth-stage technology company focused on transforming the healthcare payer industry, creating a more personalized patient experience, improving health outcomes, and lowering the overall cost of healthcare. Abacus Insights provides a flexible, efficient, and secure platform that organizes and exchanges healthcare data from various sources and formats, allowing our customers to uncover differentiated insights that address their clients' needs. Our employees know that they play an active role in keeping our customers' data safe and are responsible for ensuring that our comprehensive policies and practices are met. With our deep expertise in cloud-ena...

Abacus Insights is a mission-driven, rapid growth-stage technology company focused on transforming the healthcare payer industry, creating a more personalized patient experience, improving health outcomes, and lowering the overall cost of healthcare. Abacus Insights provides a flexible, efficient, and secure platform that organizes and exchanges healthcare data from various sources and formats, allowing our customers to uncover differentiated insights that address their clients' needs. Our employees know that they play an active role in keeping our customers' data safe and are responsible for ensuring that our comprehensive policies and practices are met. With our deep expertise in cloud-ena...

Job Summary: Skills : Lead Data Eng with Scala, Spark, Pyspark & Databricks Job Title : Data Engineer (Scala, Spark, Pyspark, Databricks) Location: Chennai Responsibilities : Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform. Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data quality, integrity, and efficiency. Optimize Spark applications for performance and cost-efficiency within the Databricks environment. Work with Delta Lake for building reliable data lakes and data warehouses, ensuring ACID transactions and data versioning. Collaborate with data scientists, analysts, and other ...

Description Must have skills: Python azure data factory Databricks Apache Spark / PySpark Azure Data Lake Store Azure Data Lake Analytics Pipeline Execution/ETL Machine Learning AI ETL clould data pipelines Docker Kubernetes AWS Requirements About the Role We are seeking a talented and passionate Machine Learning Engineer to join our growing team. In this role, you will be responsible for developing and implementing machine learning models to solve real-world problems and drive business value. You will work closely with data scientists, software engineers, and product managers to design, build, and deploy innovative solutions. Key Responsibilities Design, develop, and implement machine learn...

Join us as a Data Engineer at Barclays, where you'll take part in the evolution of our digital landscape, driving innovation and excellence. You'll harness cutting-edge technology to revolutionize our digital offerings, ensuring unparalleled customer experiences. As a part of the team, you will deliver technology stack, using strong analytical and problem solving skills to understand the business requirements and deliver quality solutions. You'll be working on complex technical problems that will involve detailed analytical skills and analysis. This will be done in conjunction with fellow engineers, business analysts and business stakeholders. To be successful as a Data Engineer you should h...

Key Responsibilities: Operate: Responsible for the operation of the system and record any problems that occur. Resolve: Ensure rapid problem resolution in close cooperation with participating third parties. Follow: Proactively follow the solution progress of incidents. Monitor: Responsible for the monitoring and analysis of possible sources of error. Update: Keep stakeholders up-to-date on system-relevant incidents. Process: Follow established processes such as Incident Management and Change Request Management. Evangelize: Promote best practices and methodologies. Automate: Identify opportunities to automate processes and procedures. Cultivate: Build relationships with developers, engineers,...

Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title And Summary Lead Data Engineer Overview We are the global technology company behind the world’s fastest payments processing network. We are a vehicle for commerce, a connection to financ...

JOB DESCRIPTION Data Engineer Designation – Data Engineer Experience – 5+ Years Location Mumbai (onsite) Job Summary: We are seeking a highly skilled Data Engineer with deep expertise in Apache Kafka integration with Databricks, structured streaming, and large-scale data pipeline design using the Medallion Architecture. The ideal candidate will demonstrate strong hands-on experience in building and optimizing real-time and batch pipelines, and will be expected to solve real coding problems during the interview. Job Description: • Design, develop, and maintain real-time and batch data pipelines in Databricks. • Integrate Apache Kafka with Databricks using Structured Streaming. • Implement rob...

Technical Skills Required: ETL Concepts: Strong understanding of Extract, Transform, Load (ETL) processes Ability to design, develop, and maintain robust ETL pipelines Database Fundamentals: Proficiency in working with relational databases (eg, MySQL, PostgreSQL, Oracle, or MS SQL Server) Knowledge of database design and optimization techniques Basic Data Visualization: Ability to create simple dashboards or reports using visualization tools (eg, Tableau, Power BI, or similar) Query Optimization: Expertise in writing efficient, optimized queries to handle large datasets Testing and Documentation: Experience in validating data accuracy and integrity through rigorous testing Ability to documen...

Description Must have skills: Python azure data factory Databricks Apache Spark / PySpark Azure Data Lake Store Azure Data Lake Analytics Pipeline Execution/ETL Machine Learning AI ETL clould data pipelines Docker Kubernetes AWS Requirements About the Role We are seeking a talented and passionate Machine Learning Engineer to join our growing team. In this role, you will be responsible for developing and implementing machine learning models to solve real-world problems and drive business value. You will work closely with data scientists, software engineers, and product managers to design, build, and deploy innovative solutions. Key Responsibilities Design, develop, and implement machine learn...

Job Description: About Us At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. Responsible Growth is how we run our company and how we deliver for our clients, teammates, communities and shareholders every day. One of the keys to driving Responsible Growth is being a great place to work for our teammates around the world. We’re devoted to being a diverse and inclusive workplace for everyone. We hire individuals with a broad range of backgrounds and experiences and invest heavily in our teammates and their families by offering competitive benefits to support their physical, emotional, and financial well-being. Bank of...

Job Title: Big Data Engineer (Azure) Location: Hyderabad / Chennai / Bangalore (Hybrid – 2 days office) Employment Type: Full-Time About the Role We’re looking for a skilled Big Data Engineer (Azure) with hands-on experience in building and optimizing large-scale data pipelines and systems. The ideal candidate has strong SQL skills, expertise in Azure data services, and the ability to work with complex datasets to deliver business insights. Key Skills & Experience 5+ years of experience in Big Data Engineering . Strong in SQL , data modeling, and handling large, messy datasets . Expertise with Hadoop, Hive, Spark, Kafka, Pig , and Databricks . Proficient with Azure Data Lake, Synapse, Data F...

Overview Analyzes, develops, designs, and maintains software for the organization's products and systems. Performs system integration of software and hardware to maintain throughput and program consistency. Develops, validates, and tests: structures and user documentation. Work may be reviewed for accuracy and overall adequacy. Follows established processes and directions. Responsibilities Design, develop and deploy core components of our next generation cloud robotics platform. Work closely with the product and other engineering teams to envision and drive the technical roadmap for cloud robotics platform Influence larger team's technical and collaborative culture and processes by growing a...

About Trademo Trademo is a Global Supply Chain Intelligence SaaS Company, headquartered in Palo-Alto, US. Trademo collects public and private data on global trade transactions, sanctioned parties, trade tariffs, ESG and other events using its proprietary algorithms.Trademo analyzes and performs advanced data processing on billions of data points (50Tb+) using technologies like Graph Databases, Vector Databases, ElasticSearch, MongoDB, NLP and Machine Learning (LLMs) to build end-to-end visibility on Global Supply Chains. Trademo's vision is to build a single truth on global supply chains to different stakeholders in global supply chains - discover new commerce opportunities, ensure complianc...

Job Overview: We are looking for a skilled and motivated Lead Data Engineer with strong experience in Python programming and Google Cloud Platform (GCP) to join our data engineering team. The ideal candidate will be responsible for requirements gathering, designing, architecting the solution, developing, and maintaining robust and scalable ETL (Extract, Transform, Load) & ELT data pipelines. The role involves working with customers directly, gathering requirements, discovery phase, designing, architecting the solution, using various GCP services, implementing data transformations, data ingestion, data quality, and consistency across systems, and post post-delivery support. Experience Level: ...

About Us Optiply is at the forefront of three rapidly expanding sectors: Software as a Service, Artificial Intelligence, and E-commerce. With our intelligent purchasing software, we empower over 300 web shops and wholesalers to make smarter buying decisions, using predictive analytics to optimize inventory management. We're not just building a product; we're building the future of autonomous inventory management. Job Description At Optiply, we run millions of demand forecasts every day to optimize the global e-commerce supply chain. As our Time Series Forecasting Engineer, you won't just be building models; you'll be engineering the core intelligence of a system that operates at a massive sc...

About Us Optiply is at the forefront of three rapidly expanding sectors: Software as a Service, Artificial Intelligence, and E-commerce. With our intelligent purchasing software, we empower over 300 web shops and wholesalers to make smarter buying decisions, using predictive analytics to optimize inventory management. We're not just building a product; we're building the future of autonomous inventory management. Job Description At Optiply, we run millions of demand forecasts every day to optimize the global e-commerce supply chain. As our Predictive Intelligence Engineer , you won't just be building models; you'll be engineering the core intelligence of a system that operates at a massive s...

3-5+ years of programming experience in coding, object-oriented design, and/or functional programming, including Python or a related language, 2+ years of experience working in distributed systems like Airflow, BigQuery, Spark, Kafka Eco System

Company Description Experian is a global data and technology company, powering opportunities for people and businesses around the world. We help to redefine lending practices, uncover and prevent fraud, simplify healthcare, provide digital marketing solutions, and gain deeper insights into the automotive market, all using our unique combination of data, analytics and software. We also assist millions of people to realize their financial goals and help them to save time and money. We operate across a range of markets, from financial services to healthcare, automotive, Agri finance, insurance, and many more industry segments. We invest in experienced people and new advanced technologies to unl...

Job Description Company: The Infinity Canvas Location: Chennai | Type: Full-time, On-site Contact: Piriyadharisini, HR – 7904018428 Email: hr@infinitycanvas.in About the Role- LinkedIn Strategist We’re not looking for someone who just “posts on LinkedIn.”We’re looking for someone who can own the feed. Someone who knows how to turn a plain thought into a viral conversation, a carousel into a client magnet, and a founder’s story into a brand people actually care about. If you understand how the algorithm breathes, when to drop a mic-worthy post, and how to make audiences stop scrolling and start thinking — this is your zone. Bring your storytelling, your strategy, and that spark that makes peo...