Device restarted (uptime low)-cisco-nxos

Device restarted (uptime low)-cisco-nxos

Vendor: cisco

OS: nxos

Description:
Indeni will alert when a device has restarted.

Remediation Steps:
Determine why the device was restarted.
|
|1. Use the “show version” or “show system reset-reason” NX-OS commands to display the reason for the reload.
|2. Use the “show cores” command to determine if a core file was recorded during the unexpected reboot.
|3. Run the “show process log” command to display the processes and if a core was created.
|4. With the show logging command, review the events that happened close to the time of reboot.

How does this work?
This script logs in to the Cisco Nexus switch using SSH and retrieves the output of the “show version” command. The output includes the device’s uptime as well as additional hardware and software related details.

Why is this important?
Capture the uptime of the device. If the uptime is lower than the previous sample, the device must have reloaded.

Without Indeni how would you find this?
The administrator would have to manually log into the Nexus switch and run the “show version” NX-OS command to review the uptime and the last time/reason that it has been reloaded.

nexus-show-version

name: nexus-show-version
description: Nexus show version
type: monitoring
monitoring_interval: 5 minutes
requires:
    vendor: cisco
    os.name: nxos
comments:
    uptime-milliseconds:
        why: |
            Capture the uptime of the device. If the uptime is lower than the previous sample, the device must have reloaded.
        how: |
            This script logs in to the Cisco Nexus switch using SSH and retrieves the output of the "show version" command. The output includes the device's uptime as well as additional hardware and software related details.
        without-indeni: |
            The administrator would have to manually log into the Nexus switch  and run the "show version" NX-OS command to review the uptime and the last time/reason that it has been reloaded.
        can-with-snmp: true
        can-with-syslog: true
    vendor:
        why: |
            Capture the device vendor name.
        how: |
            This script logs in to the Cisco Nexus switch using SSH and retrieves the output of the "show version" command. The output includes the device's hardware and software related details.
        without-indeni: |
            The administrator would have to manually log into the Nexus switch and run the "show version" NX-OS command to capture hardware/software information such as vendor name, OS name, OS version and hostname.
        can-with-snmp: true
        can-with-syslog: false
    os-name:
        why: |
            Capture the device operating system name.
        how: |
            This script logs in to the Cisco Nexus switch using SSH and retrieves the output of the "show version" command. The output includes the device's hardware and software related details.
        without-indeni: |
            The administrator would have to manually log into the Nexus switch and run the "show version" NX-OS command to capture hardware/software information such as vendor name, OS name, OS version and hostname.
        can-with-snmp: true
        can-with-syslog: false
    os-version:
        why: |
            Capture the device operating system version. The version should be the same across all members of a vPC (cluster).
        how: |
            This script logs in to the Cisco Nexus switch using SSH and retrieves the output of the "show version" command. The output includes the device's hardware and software related details.
        without-indeni: |
            The administrator would have to manually log into the Nexus switch and run the "show verion" NX-OS command to capture hardware/software information such as vendor name, OS name, OS version and hostname.
        can-with-snmp: true
        can-with-syslog: false
    hostname:
        why: |
            Capture the device hostname.
        how: |
            This script logs in to the Cisco Nexus switch using SSH and retrieves the output of the "show version" command. The output includes the device's hardware and software related details.
        without-indeni: |
            The administrator would have to manually log into the Nexus switch and run the "show version" NX-OS command to capture hardware/software information such as vendor name, OS name, OS version and hostname.
        can-with-snmp: true
        can-with-syslog: false
steps:
-   run:
        type: SSH
        command: show version
    parse:
        type: AWK
        file: show_version.parser.1.awk

cross_vendor_uptime_low

// Deprecation warning : Scala template-based rules are deprecated. Please use YAML format rules instead.

package com.indeni.server.rules.library.templatebased.crossvendor

import com.indeni.apidata.time.TimeSpan
import com.indeni.apidata.time.TimeSpan.TimePeriod
import com.indeni.server.common.data.conditions.Equals
import com.indeni.server.rules.RuleContext
import com.indeni.server.rules.library.templates.TimeIntervalThresholdOnDoubleMetricTemplateRule
import com.indeni.server.sensor.models.managementprocess.alerts.dto.AlertSeverity
import com.indeni.server.rules.ThresholdDirection
import com.indeni.server.rules.RemediationStepCondition

case class cross_vendor_uptime_low() extends TimeIntervalThresholdOnDoubleMetricTemplateRule(
  ruleName = "cross_vendor_uptime_low",
  ruleFriendlyName = "All Devices (Non-VSX): Device restarted (uptime low)",
  ruleDescription = "Indeni will alert when a device has restarted.",
  severity = AlertSeverity.CRITICAL,
  metricName = "uptime-milliseconds",
  threshold = TimeSpan.fromMinutes(60),
  metricUnits = TimePeriod.MILLISECOND,
  thresholdDirection = ThresholdDirection.BELOW,
  alertDescriptionFormat = "The current uptime is %.0f seconds which seems to indicate the device has restarted.",
  alertDescriptionValueUnits = TimePeriod.SECOND,
  baseRemediationText = "Determine why the device was restarted.",
  metaCondition = !Equals("vsx", "true")
)(
  RemediationStepCondition.VENDOR_CISCO ->
    """|
       |1. Use the "show version" or "show system reset-reason" NX-OS commands to display the reason for the reload.
       |2. Use the "show cores" command to determine if a core file was recorded during the unexpected reboot.
       |3. Run the "show process log" command to display the processes and if a core was created.
       |4. With the show logging command, review the events that happened close to the time of reboot.""".stripMargin,
  RemediationStepCondition.VENDOR_FORTINET ->
    """
      |1. Watch the system reboot time.
      |2. Review the log messages and focus on error messages that were generated at least 5 minutes prior to system reboot, especially before unexpected system reboot.
      |3. Verify the status of the scheduled restart command to making  sure it's an irregular restart
      |   - config sys global
      |   - get | grep restart
      |   - end
      |4. Login via ssh to the Fortinet firewall and review the crash log in a readable format by using the FortiOS command “diag debug crashlog read”.
      |5. Contact Fortinet Technical support at https://support.fortinet.com/ for further assistance.""".stripMargin
)