Setting Up Splunk on Ubuntu for AWS Log Management

In today’s digital landscape, effective log management is crucial for maintaining security and performance in cloud environments. This post outlines the step-by-step process of setting up a local Splunk Enterprise server on Ubuntu and integrating it with AWS VPC Flow Logs. By following this guide, you will learn how to ingest, index, and analyze cloud logs, enabling you to harness the power of Splunk for log data analytics in a controlled, local setting. Whether you’re interested in enhancing your security expertise or conducting practical learning exercises, this setup provides a valuable resource for understanding log management in a cloud-connected world.

This setup demonstrates how cloud logs can be ingested into a SIEM-style platform for local searching and analysis.

Download Splunk Enterprise

First, download the Splunk Enterprise Linux package. In this setup, I used the following command to download the package directly:

wget -O splunk.tgz https://download.splunk.com/products/splunk/releases/9.2.1/linux/splunk-9.2.1-78803f08aabb-Linux-x86_64.tgz

This saves the downloaded archive locally as splunk.tgz.

Install Splunk on Ubuntu

After downloading the package, extract it using the following command:

tar -xvzf splunk.tgz

Move the extracted Splunk directory to /opt:

sudo mv splunk /opt/

Now start Splunk for the first time and accept the licence agreement:

sudo /opt/splunk/bin/splunk start --accept-license

Create the Splunk Administrator Account

When Splunk starts for the first time, it prompts you to create an administrator account. This account is required to access the Splunk web interface.

Enter a username when prompted. For example:

admin

Then enter and confirm a password. The password will not be visible while typing, which is expected behaviour in the terminal.

Once the administrator account is created, Splunk completes its startup process and provides the URL for accessing the web interface.

Open Splunk in the Browser

Open a browser and go to:

http://localhost:8000

Log in using the administrator credentials created during the first start.

Export AWS VPC Flow Logs

For this example, AWS VPC Flow Logs had already been configured to send network flow records to CloudWatch Logs. To bring those logs into Splunk, I first exported them from CloudWatch Logs using the AWS CLI.

aws logs filter-log-events \
  --region eu-west-2 \
  --log-group-name "/vpc/flowlogs/security-lab" \
  --output text > vpc_flow_logs.txt

After exporting the logs, verify that the local file has been created:

ls -lh vpc_flow_logs.txt

In my case, the file was created successfully and contained the exported flow log data.

Add Data to Splunk

From the Splunk home page, click Add data. This starts the data ingestion workflow.

Splunk provides multiple ways to bring data in. For this local setup, select Upload, because the VPC Flow Logs have already been exported to a local text file.

Select the Source File

In the Select Source step, choose the exported log file:

vpc_flow_logs.txt

After selecting the file, click Next.

Preview the Log Data

Splunk now displays a preview of the uploaded file. This step is useful because it confirms that Splunk can read the file and split the content into separate events.

The VPC Flow Log entries contain details such as account ID, network interface ID, source IP, destination IP, source port, destination port, protocol, packets, bytes, action and log status.

Create a Custom Source Type

To identify these logs clearly inside Splunk, I saved a custom source type.

Click Save As and enter the following source type name:

aws_vpc_flow_logs

This makes it easier to search for the imported VPC Flow Logs later.

Configure Input Settings

In the Input Settings step, Splunk asks for the host and index configuration.

The host value can remain as the local Ubuntu machine name. For the index, select:

main

For a local test setup, using the main index is sufficient. In a production setup, it is usually better to create a separate index for AWS logs.

Review and Submit

On the review screen, Splunk shows the selected file, source type, host and index. Review the settings before submitting.

In this setup, the review screen showed:

• Input Type: Uploaded File
• File Name: vpc_flow_logs.txt
• Source Type: aws_vpc_flow_logs
• Host: local Ubuntu machine
• Index: main

Click Submit to complete the upload.

Confirm Successful Upload

After submission, Splunk confirms that the file has been uploaded successfully. The VPC Flow Logs are now available inside Splunk for searching and analysis.

Basic Searches

After ingestion, open the Search & Reporting app to search the imported logs.

To view the imported VPC Flow Logs:

index=main sourcetype=aws_vpc_flow_logs

To search for rejected traffic:

index=main sourcetype=aws_vpc_flow_logs REJECT

To search for accepted traffic:

index=main sourcetype=aws_vpc_flow_logs ACCEPT

These basic searches are enough to confirm that the AWS logs have been ingested and can be searched inside Splunk.

Useful Splunk Commands

Since Splunk is installed locally, it is useful to know how to stop, start and check the service status.

To stop Splunk:

sudo /opt/splunk/bin/splunk stop

To start Splunk again:

sudo /opt/splunk/bin/splunk start

To check Splunk status:

sudo /opt/splunk/bin/splunk status

Conclusion

In this post, I installed Splunk Enterprise locally on Ubuntu and imported AWS VPC Flow Logs into it. This provided a simple local SIEM-style environment where AWS network logs could be uploaded, indexed and searched.

This setup is useful for cloud security labs, learning exercises and quick demonstrations where a full production ingestion pipeline is not required.