Job Description

Strong knowledge of Splunk architecture, components, and deployment models (standalone, distributed, or clustered) Hands-on experience with Splunk forwarders, search processing, and index clustering Proficiency in writing SPL (Search Processing Language) queries and creating dashboards Familiarity with Linux/Unix systems and basic scripting (e.g., Bash, Python) Understanding of networking concepts and protocols (TCP/IP, syslog) We are looking for a Splunk architect to join our dynamic team. In this hybrid role, you will leverage your expertise in Python programming to develop innovative solutions while harnessing the power of Splunk for data analysis, monitoring, and automation. This position is ideal for a problem-solver passionate about integrating programming with operational intelligence tools to drive efficiency and insights across the organization. Key Responsibilities Deploy Splunk Enterprise or Splunk Cloud on servers or virtual environments. Configure indexing and search head clusters for data collection and search functionalities. Deploy universal or heavy forwarders to collect data from various sources and send it to the Splunk environment Configure data inputs (e.g., syslogs, snmp, file monitoring) and outputs (e.g., storage, dashboards) Identify and onboard data sources such as logs, metrics, and events. Use regular expressions or predefined methods to extract fields from raw data Configure props.conf and transforms.conf for data parsing and enrichment. Create and manage indexes to organize and control data storage. Configure roles and users with appropriate permissions using role-based access control (RBAC). Integrate Splunk with external authentication systems like LDAP, SAML, or Active Directory Monitor user activities and changes to the Splunk environment Optimize Splunk for better search performance and resource utilization Regularly monitor the status of indexers, search heads, and forwarders Configure backups for configurations and indexed data Diagnose and resolve issues like data ingestion failures, search slowness, or system errors. Install and manage apps and add-ons from Splunkbase or custom-built solutions. Create python scripts for automation and advanced data processing. Use KV stores for dynamic data storage and retrieval within Splunk Plan and execute Splunk version upgrades Regularly update apps and add-ons to maintain compatibility and security Ensure the underlying operating system and dependencies are up-to-date. Integrate Splunk with ITSM tools (e.g., ServiceNow), monitoring tools, or CI/CD pipelines. Use Splunk's REST API for automation and custom integrations Good to have Splunk Core Certified Admin certification Splunk Development and Administration Build and optimize complex SPL (Search Processing Language) queries for dashboards, reports, and alerts. Develop and manage Splunk apps and add-ons, including custom Python scripts for data ingestion and enrichment. Onboard and validate data sources in Splunk, ensuring proper parsing, indexing, and field extractions. Integration and Automation Leverage Python to automate Splunk administrative tasks such as monitoring, data onboarding, and alerting. Integrate Splunk with third-party tools, systems, and APIs (e.g., ServiceNow, cloud platforms, or in-house solutions). Develop custom connectors to stream data between Splunk and other platforms or databases. Data Analysis and Insights Collaborate with stakeholders to extract actionable insights from log data and metrics using Splunk. Create advanced visualizations and dashboards to highlight key trends and anomalies. Assist in root cause analysis for performance bottlenecks or operational incidents. System Optimization and Security Enhance Splunk search performance through Python-driven optimizations and configurations. Implement security best practices in both Python code and Splunk setups, ensuring compliance with regulatory standards. Perform regular Splunk system health checks and troubleshoot issues related to data ingestion or indexing. Collaboration and Mentoring Work closely with DevOps, Security, and Data teams to align Splunk solutions with business needs. Mentor junior developers or administrators in Python and Splunk best practices. Document processes, solutions, and configurations for future reference. Python Development: Proficient in Python 3.x, with experience in libraries such as Pandas, NumPy, Flask/Django, and Requests. Strong understanding of RESTful APIs and data serialization formats (JSON, XML). Experience with version control systems like Git. Design, develop, and maintain robust Python scripts, applications, and APIs to support automation, data processing, and integration workflows. Create reusable modules and libraries to simplify recurring tasks and enhance scalability. Debug, optimize, and document Python code to ensure high performance and maintainability. Splunk Expertise: Hands-on experience in Splunk development, administration, and data onboarding. Proficiency in SPL (Search Processing Language) for creating advanced searches, dashboards, and alerts. Familiarity with props.conf and transforms.conf configurations. Other Skills: Knowledge of Linux/Unix environments, including scripting (Bash/PowerShell). Understanding of networking protocols (TCP/IP, syslog) and log management concepts. Experience with cloud platforms (AWS, Azure, or GCP) and integrating Splunk in hybrid environments. Show more Show less