4 Pyspark Coding Jobs

As a Databricks Architect at our organization, you will be responsible for engaging in a diverse range of impactful customer technical Big Data projects. This includes developing reference architectures, how-to guides, and Minimal Viable Products (MVPs). Your role will involve leading strategic initiatives that encompass the end-to-end design, build, and deployment of industry-leading big data and AI applications. One of your key responsibilities will be to provide architectural guidance that fosters the adoption of Databricks across business-facing functions. You will collaborate with platform engineering teams to ensure the effective implementation of Databricks services within our infrastructure. Additionally, you will evaluate and assess new features and enhancements on the Databricks platform to ensure we leverage the latest capabilities. Your role will also involve designing and implementing various integration patterns involving Databricks and third-party tools such as Collibra and data quality solutions. You will utilize your expertise in AWS administration and architecture to optimize cloud resources and ensure seamless integration with Databricks. Furthermore, leveraging your hands-on experience with Databricks Unity Catalog, you will implement robust data governance and lineage capabilities. Advocating for and implementing CI/CD practices to streamline the deployment of Databricks solutions will be an essential part of your responsibilities. You will also contribute to the development of a data mesh architecture, promoting decentralized data ownership and accessibility across the organization. Understanding and articulating the analytics capabilities of Databricks will enable teams to derive actionable insights from their data. An ideal candidate for this role would have 7+ years of experience with Big Data technologies, including Apache Spark, cloud native data lakes, and data mesh platforms, in a technical architecture or consulting role. You should also possess 5+ years of independent experience in Big Data architecture and proficiency in coding with Python. Extensive experience working with AWS cloud platforms, including a solid understanding of AWS services and architecture, is required. Strong documentation and whiteboarding skills are essential to effectively communicate complex ideas. You should have in-depth knowledge of the latest services offered by Databricks and the ability to evaluate and integrate these services into our platform. Proven experience in implementing solutions using Databricks Unity Catalog, with a focus on data governance and lineage tracking, is highly desirable. Demonstrated expertise in migrating from Databricks classic platform to Lakehouse architecture, utilizing Delta file format and/or Delta Live Tables, will be an advantage. A collaborative mindset with the ability to work effectively across teams and functions is crucial for success in this role.,

We are a technology-led healthcare solutions provider driven by our purpose to enable healthcare organizations to be future-ready. Our goal is to offer accelerated, global growth opportunities for talent that is bold, industrious, and nimble. At Indegene, you will gain a unique career experience that celebrates entrepreneurship and is guided by passion, innovation, collaboration, and empathy. If you are looking to explore exciting opportunities at the convergence of healthcare and technology, visit www.careers.indegene.com. Are you ready to jump-start your career We understand the importance of the first few years of your professional journey, which lay the foundation for your entire career. At Indegene, we promise you a differentiated career experience where you will work at the exciting intersection of healthcare and technology and be mentored by industry experts. We offer a global fast-track career path that aligns with Indegene's high-speed growth. We are purpose-driven, enabling healthcare organizations to be future-ready, with customer obsession as our driving force. Our approach is bold in actions, nimble in decision-making, and industrious in our work ethic. If this excites you, then apply for the role of Senior Data Analyst at Indegene. Overview: We are seeking a dynamic and experienced Senior Data Analyst to join our team. As a Senior Data Analyst, you will play a pivotal role in creating and designing machine learning (ML) and analytics solutions in the pharmaceutical analytics domain. Key Responsibilities: - Collaborate with managers and onshore counterparts to assess client requirements and objectives, including implementing Omnichannel strategy - Analyze large and complex datasets related to sales, call plans, market share, patient outcomes, field segmentation, and more - Conduct ad hoc analyses to address specific business questions from cross-functional teams - Ensure data quality, accuracy, and consistency through regular quality checks, test cases, and validations - Utilize statistical and machine learning techniques to build predictive models (e.g., patient journey analysis, sales forecasting, MMx modeling) - Guide junior analysts and provide technical mentorship, especially in MLOps using DSS, ETL using Snowflake, and Pyspark coding Desired Profile - Qualifications: - Bachelor's or Master's degree in Computer Science, Data Science, or Pharmaceutical Sciences - 5 years of experience in leading analytical projects in the pharma domain - 3 years of experience in advanced analytics, including AI, machine learning, predictive modeling, advanced analytics, and statistical modeling - Experience in pharma commercial analytics is a plus - Proven track record of successfully implementing Omnichannel strategy or similar technologies in the pharmaceutical or related industry - Strong understanding of the pharma ecosystem, key data sources, industry trends, and compliance requirements - Excellent understanding of machine learning techniques and algorithms - Proficiency in data science techniques and tools including Python, SQL, PySpark, Dataiku, AWS cloud services, Git, Tableau, PowerBI, etc. - Experience in MLOps is an advantage - Strong analytical and problem-solving abilities with attention to detail - Willingness to work in a highly demanding and result-oriented team environment - Excellent communication skills to convey complex ideas clearly This is an equal opportunity position based in Bangalore, KA, IN.,

You will be responsible for architecting data warehousing and business intelligence solutions to address cross-functional business challenges. This will involve interacting with business stakeholders to gather requirements and deliver comprehensive Data Engineering, Data Warehousing, and analytics solutions. Additionally, you will collaborate with other technology teams to extract, transform, and load data from diverse sources. You should have a minimum of 5-8 years of end-to-end Data Engineering Development experience, preferably across industries such as Retail, FMCG, Manufacturing, Finance, Oil & Gas. Experience in functional domains like Sales, Procurement, Cost Control, Business Development, and Finance is desirable. You are expected to have 3 to 10 years of experience in data engineering projects using Azure or AWS services, with hands-on expertise in data transformation, processing, and migration using various tools such as Azure Data Lake Storage, Azure Data Factory, Databricks, AWS Glue, Redshift, and Athena. Familiarity with MS Fabric and its components will be advantageous, along with experience in working with different source/target systems like Oracle Database, SQL Server Database, Azure Data Lake Storage, ERP, CRM, and SCM systems. Proficiency in reading data from sources via APIs/Web Services and utilizing APIs to write data to target systems is essential. You should also have experience in Data Cleanup, Data Cleansing, and optimization tasks, including working with non-structured data sets in Azure. Knowledge of analytics tools like Power BI and Azure Analysis Service, as well as exposure to private and public cloud architectures, will be beneficial. Excellent written and verbal communication skills are crucial for this role. Ideally, you hold a degree in M.Tech / B.E. / B.Tech (Computer Science, Information systems, IT) / MCA / MCS. Key requirements include expertise in MS Azure Data Factory, Python, PySpark Coding, Synapse Analytics, Azure Function Apps, Azure Databricks, AWS Glue, Athena, Redshift, and Databricks Pysark. Exposure to integration with various applications/systems like ERP, CRM, SCM, WebApp using APIs, Cloud, On-premise systems, DBs, and file systems is expected. The role necessitates a minimum of 3 Full Cycle Data Engineering Implementations (5-10 years of experience) with a focus on building data warehouses and implementing data models. Exposure to the consulting industry is mandatory, along with strong verbal and written communication skills. Your primary skills should encompass Data Engineering Development, Cloud Engineering with Azure or AWS, Data Warehousing & BI Solutions Architecture, Programming (Python PySpark), Data Integration across various systems, Consulting experience, ETL and Data Transformation, and knowledge in Cloud Architecture. Additionally, familiarity with MS Fabric, handling non-structured data, Data Cleanup and Optimization, API/Web Services, Data Visualization, and industry and functional knowledge will be advantageous. The compensation package ranges from INR 12-28 lpa, subject to the candidate's performance and experience level.,

Role: Data Engineer Experience: 6-7 Years Location: Bangalore, Pune Notice Period: Immediate Joiners Only Role & responsibilities : Mandate Skills Strong - Pyspark (Programming) Databricks Background and experience: Bachelors Degree in Computer Science or related field 6-7 years of experience in Cloud Data Engineering. Technical and professional skills: We are looking for a flexible, fast learning, technically strong Data Engineer. Expertise is required in the following fields: Proficient in Cloud Services Azure Build ETL and data movement solutions. Migrate data from traditional database systems to Cloud environment Strong hands-on experience for working with Streaming dataset Building Complex Notebook in Databricks to achieve business Transforma tions. Hands-on Experience in Data Refinement using Pyspark and Spark SQL Familiarity with building dataset using Scala. Familiarity with tools such as Jira and GitHub Experience leading agile scrum, sprint planning and review sessions Good communication and interpersonal skills Reach us:If you are interested in this position and meet the above qualifications, please reach out to me directly at swati@cielhr.com and share your updated resume highlighting your relevant experience.