Nazaudy, a spark in your curious mind

Splunk Enterprise and VMware & NetApp monitoring

Let's say your have Splunk Enterprise and VMware & NetApp monitoring, and you want to configure Splunk to gather date from VMware and NetApp, what do you do? Let's investigate the procedure on this article

 The steps are as follows:

  1. Setup the Splunk OVA for VMware
  2. Redirect logs on your vCenter to the DCN, and open firewall
  3. redirect logs on your ESXi hosts to the DCN, and open firewall
  4. Install the "Splunk Add-on for VMware" on Splunk Enterprise
  5. Install the "Splunk App for VMware" on Splunk Enterprise

Update Feb 2022: This article is only for references purposes. these apps have reached the end of life and Splunk does not longer distribute them

1. Setup the Splunk OVA for VMware

Download this OVA on your vSphere and start it up: https://splunkbase.splunk.com/app/3216/ (at the time of writing the version of OVA they have online is 3.4.1)

Splunk OVA for VMware

 

After power up, logon with "root" and "changemenow", then run the DCN (Data Collection Node) network configuration utility

DCN Network Config

On my example I called the DCN "Splunk_Collector", and gave it an IP of 10.10.10.33, don't forget as well to add an entry on DNS of your "Splunk_Collector" IP, it will need it

 Splunk Collector DNS

And now that you are on DNS, ensure there is an entry as well for your Enterprise Splunk server, on my case I called it "heappsvr3" with the IP 10.10.10.130

 

Once all this is done, run the "dcn-splunk-config" to configure Splunk by following the wizard

  • For your indexers enter: 10.10.10.130:8089
  • For your license master enter: https://heappsvr3

DCN-Splunk-Config

Once you got the IP address configured, you can access the VM if you like by visiting https://10.10.10.33:8000 , but DO NOT change anything yet

 

passwd root  //** use this command to change the default root password

  

2. Redirect logs on your vCenter to the DCN, and open firewall

Now we need to visit our vCenter (on my example I'm running the appliance, so it is a VCSA) and also the ESXi hosts, and configure all to send their logs to the Data Collector Node

If you are running VCSA 6.0 visit System Configuration > Nodes > Related Objects and find the VMware Syslog Service, and configure this service accordingly. 

On my example I'm runing VCSA 6.5, so the SysLog configuration for this version are kept under the VAMI ui https://10.10.10.144:5480

VCSA for Splunk Enterprise and VMware & NetApp monitoring

Visit also your vCenter  > Configure > Advanced Settings and set both the "config.log.outputToSyslog" and "config.log.outputToSyslog" to true

vCenter-Syslog-settings

After modifying this configuration, you need to restart the VCSA or its vCenter service

 Once your VCSA comes back online, visit System Configuration > Nodes > Manage > Firewall and white-list both IPs for your Splunk Enterprise and your Data Collector Node

VCSA-firewall 

 

3. Redirect logs on your ESXi hosts to the DCN, and open firewall

Now we need to do the same thing with the ESXi hosts; for these guys visit the Configure tab > Advanced System Settings and configure the setting "Syslog.global.logHost" to read tcp://DCN_IP_or_DNS:1514

ESXi-syslog-settings

Do this modification on all your hosts, and after that SSH to the hosts and run this command:

esxi system syslog reload

**//run this command to check that the port of the DCN is accesible:

nc -z 10.10.10.33 514

If SSH is disable on the ESXi host, enable it by visiting Configure > Security Profile > scroll down for services, and start the SSH service

ESXi-enable-SSH

 

Configure also the firewalls on the ESXi hosts

ESXi-syslog-firewall

 

 

Enable the TCP data inputs in the Splunk Server

 

2. Install the "Splunk Add-on for VMware" on Splunk Enterprise

Next, download the VMware add-on for this site: https://splunkbase.splunk.com/app/3215/ , which at the time of writing is version 3.4.1

Splunk-Add-on-for-VMware for Splunk Enterprise and VMware & NetApp monitoring

 Unzip the contents of the add-on VMware, and copy them to the %SPLUNK_HOME%/etc/apps on your Enterprise server, then restart the Splunk service

VMware-add-on-Contents

After you logon to the Enterprise Server, you would see the new app icon on the left hand side

Add-on-for-VMware

Before attempting to launch it, visit Settings > Access Control > Users and add to the admin the "splunk_vmware_admin" role

Add-vmware-admin-role

After adding the role to the admin user, open the "Add-on for VMware", and it should load with no configuration whatsoever

DCN-Empty-config

Click the "+" symbol to create a New Collection Node, and add the details of the OVA. Notice that you also have the "VMware add-on" on the actual Splunk OVA, but don't get confused and don't configure anything there yet

Create-New-Collection-Node

 Add also the details of your vCenter until you see the configuration like below, with both the DCN and vCenter added. Then click on "Start Scheduler"

Start-Scheduler-VMware

 At this stage, you are ready to start the installation of the VMware App in your Splunk Enterprise

 

 Configure the ESXi to use To configure remote syslog using TCP on port 514: esxcli system syslog config set --loghost='tcp://10.11.12.13:514'

 

1. Install the "Splunk App for VMware" on Splunk Enterrprise

On your Splunk Enterprise (also called your Search Header, if you only have one ), visit this page and download the "Splunk App for VMware" (at the time of writing this document version 3.4.1) https://splunkbase.splunk.com/app/725/

Splunk-App-for-VMware

There is a great video tutorial to follow for the installation: https://www.youtube.com/watch?v=GgJUkh0eFH4 , but we'll do this our own way

1.First unzip the contents for the VMware App

VMware-app-contents

Copy the contents into the %SPLUNK_HOME%/etc/apps folder on your Enterprise server:

ETC-folder-in-Splunk-Enterprise

Then restart the Splunk service on the server by visiting Settings > Server Control > Restart. Once you log back in again you'd be able to see the VMware App

Start the VMware App by accepting the default setup

 Vmware-App

Click on the app and follow "Continue to app setup page", then accept the default vales and click "Save"

 VMware-App-default-settigs

 

 

 

 NET APP

For the configuration of the NetApp, please use this tutorial: http://docs.splunk.com/Documentation/AddOns/released/NetApp/Configureinputs 

If you have enjoyed this article about Splunk Enterprise and VMware & NetApp monitoring you might find this other one interesting too:

 

 

 

 

References