Guides

Splunk admin guides

Practical answers to the Splunk admin questions that come up every week, each with the SPL to run. They work with or without Drew.

• How to check Splunk license usage by index, sourcetype and host

  SPL searches to see Splunk license usage per index, sourcetype and host, daily usage against your quota, and why host and source can come back empty.

• Splunk health check: a practical checklist with searches

  A Splunk health check checklist with the SPL for each item: skipped searches, blocked queues, quiet hosts, parsing errors, license, indexes, certificates and KV store.

• Splunk skipped searches: how to find the cause and fix it

  Find which Splunk scheduled searches are skipped and why, with SPL for the skip ratio and reasons, and the fixes for concurrency limits and long-running searches.

• Splunk forwarder not sending data: a troubleshooting guide

  Step-by-step checks when a Splunk universal forwarder stops sending data: find quiet hosts, check the connection, outputs.conf, inputs, missing indexes and timestamps.

• Onboarding data in Splunk: props.conf for line breaking and timestamps

  Set props.conf so Splunk breaks events and reads timestamps right: LINE_BREAKER, SHOULD_LINEMERGE, TIME_PREFIX, TIME_FORMAT, where each goes, how to test.

• Splunk blocked queues: how to find the bottleneck

  Read Splunk's indexing pipeline queues in metrics.log, find which queue blocks first, and fix the usual causes: slow disk, heavy regex, and blocked outputs.

• Splunk KV store not starting: causes and checks

  Why the Splunk KV store fails to start and how to check: kvstore-status, mongod.log, expired certificates and CA chains, port 8191, disk space and permissions.