Nazaudy, a spark in your curious mind

Install Squid, webmin and sent data to Splunk with CentOS 7

This article explains you how to install Squid, webmin and sent data to Splunk with CentOS 7 Linux Operating System. Follow this steps to get started:

  1. Install CentOS 7 and Squid
  2. Install Webmin
  3. Install the Squid App in your Splunk server
  4. Install Linux Splunk Universal Forwarder
  5. Deploy the Splunk client for data input

 

1.- Install CentOS 7 and Squid

At the time of installing your Linux box, I personally prefer to install always the "Server GUI" version, not ideal for highly secure environment but easier to troubleshoot if you (like me)  are not a Linux-guru (yet)

 Install CentOS 7

Right after you install it, start a SSH connection to your server and issue these commands:

yum -y update

yum -y install epel-release

yum -y install squid

yum -y update

yum clean all

Enable Squid to start at system boot:

systemctl enable squid

You can use the following command to control the Squid program

systemctl start squid

systemctl status squid

systemctl restart squid

systemctl stop squid

squid -v     //**help options

squid -h     //**view the version number

 The main important location for Squid are:

  • /var/log/squid/access.log ;the log of the proxy connections that we later on need to forward to Splunk
  • /etc/squid/squid.conf ;the configuration file for the ACL lists allowed to use the Squid Proxy

 

 

2.- Install Webmin

To manage Squid through Webmin, first visit the webmin download page here (http://www.webmin.com/download.html), and then copy the shortcut for the RMP package to your SSH connection of your CentOS, and install it as follows:

wget http://prdownloads.sourceforge.net/webadmin/webmin-1.831-1.noarch.rpm

Then issue "ls" to ensure you are in the directly where the RPM has been downloaded

Install webmin using wget

//**verify your location first

yum install webmin-1.831-1.noarch.rpm

 

For CentOS 7

To install webmin on CentOS 7, do as follows:

sudo vi /etc/yum.repos.d/webmin.repo

#enter the below in the above file:
[Webmin]
name=Webmin Distribution Neutral
#baseurl=http://download.webmin.com/download/yum
mirrorlist=http://download.webmin.com/download/yum/mirrorlist
enabled=1

wget http://www.webmin.com/jcameron-key.asc
sudo rpm --import jcameron-key.asc

sudo yum install webmin

After installation, visit

Configure the firewall

Before visiting the webmin page, we need to open the default port that it uses (10,000) on the firewall. To do that, perform as follows:

sudo firewall-cmd --set-default-zone=trusted

firewall-cmd --get-default-zone

sudo firewall-cmd --zone=trusted --add-port=10000/tcp --permanent

sudo firewall-cmd --reload

 After that, visit the website https://192.168.0.127:10000 or whichever IP address you have on your CentOS

  • Username: root
  • Password: [your current root password]

Once Webmin opens, visit Servers > Squid Proxy Server and click to Initialize Cache:

 

Insall Squid

After the cache has started, click on the "Access Control" button to start with your configuration of Squid:

 Squid Proxy Server access control

Or you may find it easier to edit the /etc/squid/squid.conf file and add the ACL entries there

After this installation, we won't need the graphical interface anymore, therefore issue this command so that CentOS does not load it:

systemctl set-default multi-user.target

 

 

3.-Install the Squid App in your Splunk server

Go to your Splunk Enterprise server and download and install the "Splunk App for Splunk Enterprise"

Note, DO NOT INSTALL THIS ONE!!! === https://splunkbase.splunk.com/app/2965/

THIS ONE (great credits to Patrick Nordien from here) :) https://splunkbase.splunk.com/app/453/

 Squid App for Splunk Enterprise

If you haven't done so, setup a receiver on port 9997 (or any other one that is free) by adding this into the C:\Program Files\Splunk\etc\system\local\inputs.conf file 

[splunktcp://9997]
connection_host = ip

inputs.conf sent data to Splunk with CentOS 7

After the addition RESTART SPLUNK by visiting website > System > Server Control > Restart Splunk

 

 

4.-Install Linux Splunk Universal Forwarder

For the client Splunk Universal Forwarder, we would actually have to open Firefox on the CentOS 7 machine, logon to your Splunik account and visit this link:

From where you would download the RPM package  and save it in the downloads folder

 Install Splunk forwarder in CentOS 7

To install it, we better use yum, as always, go to the location where you downloaded the file and install it

yum -y install splunkforwarder-7.0.2-03bbabbd5c0f-linux-2.6-x86_64.rpm

  • If you don't have the GUI installed, you can copy the file using WinSCP
  • You can also use the wget command to download the installation: wget -O splunkforwarder-7.3.0-657388c7a488-linux-2.6-x86_64.rpm 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.3.0&product=universalforwarder&filename=splunkforwarder-7.3.0-657388c7a488-linux-2.6-x86_64.rpm&wget=true'

Visit /opt/splunkforwarder/bin/ and run the following to agree to the license (you can also do "./splunk start --accept-license")

./splunk

Then add the forwarder server, specifying the IP address of you Splunk Enterprise server and the port you configured on the inputs.conf file of the Splunk server

./splunk add forward-server 192.168.0.214:9997
./splunk add monitor /var/log/squid/access.log
./splunk add monitor /var/log/squid/
./splunk restart
./splunk list forward-server
./splunk list monitor
./splunk enable boot-start   //**this ensures Splunk runs at boot up

Ensure that port is open on the CentOS firewall (verify the correct zone is selected!)

firewall-cmd --get-default-zone
firewall-cmd --zone=trusted --add-port=9997/tcp --permanent
firewall-cmd --reload
firewall-cmd --runtime-to-permanet
firewall-cmd --list-all-zones  //**verify that on your ACTIVE zone port 9997 is allowed
firewall-cmd --list-ports    //**similar as above, will show you if 9997 is allowed

To verify that all configuration has been done correctly, visit the following files and DOUBLE and TRIPLE check that you have entered the right formatting:

inputs.conf

Splunk inputs.conf

outputs.conf

Splunk outputs.conf

If all goes well, you should be able to see that you Splunk Enterprise server show as active when you run the command below; if it shows as inactive make sure that you have configure your Splunk to receive data on port 9997 by visiting its web on the location Settings > Forwarding and Receiving > Receiving data

Splunk Active forwards

 

   

5.-Deploy the Splunk client for data input

 Visit again /opt/splunkforwarder/bin/ and run the following: 

./splunk set deploy-poll 192.168.0.214:8089
//**If you are asked to logon -session invalid- use the following default settings:
       Username: admin
       Password: changeme

This will create a new file called "deploymentclient.conf" under /opt/splunkforwarder/etc/system/local/ ;that file will be use to generate data input on your Splunk server.  Now RESTART your Splunk Enterprise and visit Settings > Data Inputs > Forwarder inputs > Files and Directories and click on "New", if all goes well you would be able to see the deployment installation of your CentOS machine

 Select Forwarders

On the "Select Source" section, type carefully the location of the Squid access log

Select source for Squid access log

Ensure that you search type of "squid" (added by the Squid App) and create a new index called "squid" to store the logs, then review one last time before submitting the new addition:

 Verify Squid settings in Splunk

If all goes well, you should be able to see straight away some data on the Squid App dashboard, well done!

Squid App dashboard

Finally, to change the logo of the Squid app, you can get the icon from here:

and customize for the App following these instructions:

where one of the key points is to add this stanza to the app.conf file in the /$SPLUNK_HOME/etc/apps/SplunkForSquid/static/

[install]
build = 2

Looks cool!

 Instal Splunk for Squid app

 

 

Troubleshooting

If when opening the Squid App you are presented with the error "Eventtype 'wineventlog-dns' does not exist or is disabled"; just go to Settings > Knowledge > Event Types and create it for the Squid App

 Error message Eventtype 'wineventlog-dns

Like this:

Configure Event Types

 

If you enjoyed reading this article about how to install Squid, webmin and sent data to Splunk with CentOS 7, you might like this other one too:

London 24 February 2018

 

References

A million thanks to Liptan Biswas for this great tutorial: https://hostpresto.com/community/tutorials/how-to-install-and-configure-squid-proxy-on-centos-7/

Nice job Shanker admin: https://www.thegeekdiary.com/5-useful-examples-of-firewall-cmd-command/

Useful resources from Splunk:

And better useful resources from the forums (tried and tested):