Blue Flower

Here there is how to install Squid on a CentOS 7 box, install webmin to manage Squid and send the logs to your Splunk sever, follow these steps:

  1. Install CentOS 7 and Squid
  2. Install Webmin
  3. Install the Squid App in your Splunk server
  4. Install Linux Splunk Universal Forwarder
  5. Deploy the Splunk client for data input

 

1.- Install CentOS 7 and Squid

At the time of installing your Linux box, I personally prefer to install always the "Server GUI" version, not ideal for highly secure environment but easier to troubleshoot if you (like me)  are not a Linux-guru (yet)

 

Right after you install it, start a SSH connection to your server and issue these commands:

yum -y update

yum -y install epel-release

yum -y install squid

yum -y update

yum clean all

Enable Squid to start at system boot:

systemctl enable squid

You can use the following command to control the Squid program

systemctl start squid

systemctl status squid

systemctl restart squid

systemctl stop squid

squid -v     //**help options

squid -h     //**view the version number

 The main important location for Squid are:

  • /var/log/squid/access.log ;the log of the proxy connections that we later on need to forward to Splunk
  • /etc/squid/squid.conf ;the configuration file for the ACL lists allowed to use the Squid Proxy

 

 

2.- Install Webmin

To manage Squid through Webmin, first visit the webmin download page here (http://www.webmin.com/download.html), and then copy the shortcut for the RMP package to your SSH connection of your CentOS, and install it as follows:

wget http://prdownloads.sourceforge.net/webadmin/webmin-1.831-1.noarch.rpm

Then issue "ls" to ensure you are in the directly where the RPM has been downloaded

//**verify your location first

yum install webmin-1.831-1.noarch.rpm

Configure the firewall

Before visiting the webmin page, we need to open the default port that it uses (10,000) on the firewall. To do that, perform as follows:

firewall-cmd --set-default-zone=trusted

firewall-cmd --get-default-zone

firewall-cmd --zone=trusted --add-port=10000/tcp --permanent

firewall-cmd --reload

 After that, visit the website https://192.168.0.127:10000 or whichever IP address you have on your CentOS

  • Username: root
  • Password: [your current root password]

Once Webmin opens, visit Servers > Squid Proxy Server and click to Initialize Cache:

 

After the cache has started, click on the "Access Control" button to start with your configuration of Squid:

 

Or you may find it easier to edit the /etc/squid/squid.conf file and add the ACL entries there

After this installation, we won't need the graphical interface anymore, therefore issue this command so that CentOS does not load it:

systemctl set-default multi-user.target

 

 

3.-Install the Squid App in your Splunk server

Go to your Splunk Enterprise server and download and install the "Splunk App for Splunk Enterprise"

Note, DO NOT INSTALL THIS ONE!!! === https://splunkbase.splunk.com/app/2965/

THIS ONE (great credits to Patrick Nordien from here) :) https://splunkbase.splunk.com/app/453/

 

If you haven't done so, setup a receiver on port 9997 (or any other one that is free) by adding this into the C:\Program Files\Splunk\etc\system\local\inputs.conf file 

[splunktcp://9997]
connection_host = ip

After the addition RESTART SPLUNK by visiting website > System > Server Control > Restart Splunk

 

 

4.-Install Linux Splunk Universal Forwarder

For the client Splunk Universal Forwarder, we would actually have to open Firefox on the CentOS 7 machine, logon to your Splunik account and visit this link:  https://www.splunk.com/en_us/download/universal-forwarder.html#tabs/linux , from where you would download the RPM package  and save it in the downloads folder

 

To install it, we better use yum, as always, go to the location where you downloaded the file and install it

yum -y install splunkforwarder-7.0.2-03bbabbd5c0f-linux-2.6-x86_64.rpm

Visit /opt/splunkforwarder/bin/ and run the following to agree to the license (you can also do "./splunk start --accept-license")

./splunk

Then add the forwarder server, specifying the IP address of you Splunk Enterprise server and the port you configured on the inputs.conf file of the Splunk server

./splunk add forward-server 192.168.0.214:9997

./splunk add monitor /var/log/squid/access.log

./splunk add monitor /var/log/squid/

./splunk restart

./splunk list forward-server

./splunk list monitor

./splunk enable boot-start   //**this ensures Splunk runs at boot up

Ensure that port is open on the CentOS firewall (verify the correct zone is selected!)

firewall-cmd --get-default-zone

firewall-cmd --zone=trusted --add-port=9997/tcp --permanent

firewall-cmd --reload

firewall-cmd --runtime-to-permanet

firewall-cmd --list-all-zones  //**verify that on your ACTIVE zone port 9997 is allowed
  firewall-cmd --list-ports    //**similar as above, will show you if 9997 is allowed

To verify that all configuration has been done correctly, visit the following files and DOUBLE and TRIPLE check that you have entered the right formatting:

inputs.conf

outputs.conf

If all goes well, you should be able to see that you Splunk Enterprise server show as active when you run the command below; if it shows as inactive make sure that you have configure your Splunk to receive data on port 9997 by visiting its web on the location Settings > Forwarding and Receiving > Receiving data

 

   

5.-Deploy the Splunk client for data input

 Visit again /opt/splunkforwarder/bin/ and run the following: 

 ./splunk set deploy-poll 192.168.0.214:8089


//**If you are asked to logon -session invalid- use the following default settings:
       Username: admin
       Password: changeme

This will create a new file called "deploymentclient.conf" under /opt/splunkforwarder/etc/system/local/ ;that file will be use to generate data input on your Splunk server.  Now RESTART your Splunk Enterprise and visit Settings > Data Inputs > Forwarder inputs > Files and Directories and click on "New", if all goes well you would be able to see the deployment installation of your CentOS machine

 

On the "Select Source" section, type carefully the location of the Squid access log

Ensure that you search type of "squid" (added by the Squid App) and create a new index called "squid" to store the logs, then review one last time before submitting the new addition:

 

If all goes well, you should be able to see straight away some data on the Squid App dashboard, well done!

Finally, to change the logo of the Squid app, you can get the icon from here: https://d33arxv7e4uhib.cloudfront.net/wp-content/uploads/2014/07/squid_logo.png  and customize for the App following these instructions: http://docs.splunk.com/Documentation/Splunk/6.2.5/AdvancedDev/AddConfigurations where one of the key points is to add this stanza to the app.conf file in the /$SPLUNK_HOME/etc/apps/SplunkForSquid/static/

[install]
build = 2

Looks cool!

 

 London, 25 February 2018

 

 

Troubleshooting

If when opening the Squid App you are presented with the error "Eventtype 'wineventlog-dns' does not exist or is disabled"; just go to Settings > Knowledge > Event Types and create it for the Squid App

 

Like this:

 London 24 February 2018

 

References:

A million thanks to Liptan Biswas for this great tutorial: https://hostpresto.com/community/tutorials/how-to-install-and-configure-squid-proxy-on-centos-7/

Nice job Shanker admin: https://www.thegeekdiary.com/5-useful-examples-of-firewall-cmd-command/

Useful resources from Splunk:

And better useful resources from the forums (tried and tested):

  

Print Friendly, PDF & Email