Senior Splunk Engineer

Everforth ECS is seeking a Senior Splunk Engineer to work in our Portland, OR office. Please Note: This position is contingent upon contract award.

 

The Senior Splunk Engineer designs, implements, maintains, and optimizes Splunk capabilities that support cybersecurity monitoring, investigation, reporting, and security operations. This role is responsible for Splunk platform engineering, data onboarding, search performance, dashboards, alerts, integrations, and technical support for SOC and cybersecurity stakeholders. 

The ideal candidate has deep hands-on experience administering Splunk Enterprise, Splunk Enterprise Security, or Splunk Cloud environments; understands security data pipelines and SIEM operations; and can independently troubleshoot complex platform, data ingestion, parsing, indexing, search, and content issues while collaborating with analysts, engineers, and program leadership. 

 

Key Responsibilities 

Splunk Platform Engineering & Administration 

• Administer, configure, maintain, and optimize Splunk Enterprise, Splunk Enterprise Security, Splunk Cloud, or distributed Splunk environments. 

• Support indexers, search heads, deployment servers, heavy forwarders, universal forwarders, apps, add-ons, knowledge objects, and role-based access controls. 

• Monitor platform health, availability, license utilization, data ingestion, storage, capacity, search concurrency, and overall performance. 

• Plan and execute upgrades, patches, configuration changes, backup and recovery activities, and platform maintenance in accordance with change management processes. 

Data Onboarding & Integration 

• Onboard, normalize, validate, and maintain security, infrastructure, cloud, endpoint, network, application, identity, and operational data sources. 

• Configure and troubleshoot inputs, forwarders, sourcetypes, indexes, props.conf, transforms.conf, field extractions, lookups, event types, tags, and data routing. 

• Map data to the Splunk Common Information Model and support data model acceleration, normalization, and content readiness for security analytics. 

• Integrate Splunk with security tools, ticketing systems, SOAR platforms, vulnerability tools, EDR solutions, firewalls, IDS/IPS, cloud platforms, and identity systems. 

Security Analytics & Detection Support 

• Develop, maintain, and tune SPL searches, correlation searches, alerts, dashboards, reports, notable event rules, and security monitoring use cases. 

• Partner with SOC analysts, threat hunters, threat intelligence analysts, and security engineers to translate detection requirements into reliable Splunk content. 

• Tune detections and searches to improve fidelity, reduce false positives, increase operational value, and support risk-based alerting or prioritization. 

• Support incident response and investigations by validating log availability, developing ad hoc searches, retrieving evidence, and assisting with event timelines. 

Dashboarding, Reporting & Metrics 

• Design and maintain dashboards, reports, scorecards, and visualizations for SOC operations, platform health, data coverage, compliance, and leadership reporting. 

• Track and report key Splunk metrics such as ingestion volume, license consumption, search performance, alert volume, source coverage, and data quality. 

• Automate recurring reporting and improve visibility into monitoring coverage, data source gaps, content effectiveness, and operational trends. 

• Develop executive, operational, and technical views that communicate platform status and security monitoring performance clearly and accurately. 

Troubleshooting, Optimization & Engineering Support 

• Diagnose and resolve complex Splunk issues involving ingestion delays, parsing problems, dropped data, search errors, slow dashboards, indexer performance, and app conflicts. 

• Optimize SPL, data models, summary indexes, scheduled searches, acceleration settings, storage usage, and search workloads for reliability and efficiency. 

• Support infrastructure planning, scaling, retention strategies, data lifecycle management, high availability, and disaster recovery considerations. 

• Collaborate with system administrators, network engineers, cloud teams, security engineers, and vendors to resolve technical dependencies and platform issues. 

Documentation, Standards & Continuous Improvement 

• Develop and maintain architecture diagrams, onboarding procedures, configuration standards, runbooks, troubleshooting guides, and operational documentation. 

• Support governance of index naming, source type standards, app deployment, permissions, data retention, change control, and knowledge object management. 

• Evaluate new Splunk apps, add-ons, content packs, integrations, and platform capabilities to improve security monitoring and operational efficiency. 

• Mentor junior engineers and analysts on Splunk usage, search practices, data validation, dashboard development, and platform troubleshooting.