Lab
A Cloud Guru

Working with Logstash

Your manager has asked you to set up an Elastic Stack to centralize syslog reporting. You will need to install and configure the following: * Elasticsearch * Logstash * Filebeat * Kibana Once all the services are installed, working together, and configured to startup after a system reboot, you should access Kibana over an SSH tunnel and make sure the system is working properly.

Try for free Contact sales

Path Info

Level

Intermediate

Duration

1h 0m

Published

Jul 30, 2018

Challenge

Install Elasticsearch

Install Elasticsearch with default settings:

Install Java:
```
	yum install java-1.8.0-openjdk -y
```

Import Elastic's GPG key:

	rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Download the Elasticsearch RPM:

	curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.3.rpm

Install Elasticsearch:
```
	rpm --install elasticsearch-6.2.3.rpm
```

Enable and start Elasticsearch:

	systemctl daemon-reload
	systemctl enable elasticsearch
	systemctl start elasticsearch

Challenge

Install Logstash

Install Logstash with default settings:

Import the Logstash key:

	rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Add the Logstash repo:

	vi /etc/yum.repos.d/logstash.repo

[logstash-6.x]
name=Elastic repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Install Logstash
```
yum install logstash -y
```

Enable and start Logstash:

	systemctl enable logstash
	systemctl start logstash

Challenge

Install Kibana

Install Kibana with default settings:

Download Kibana:

	curl -O https://artifacts.elastic.co/downloads/kibana/kibana-6.2.3-x86_64.rpm

Install Kibana:
```
	rpm --install kibana-6.2.3-x86_64.rpm
```

Enable and start Kibana:

	systemctl enable kibana
	systemctl start kibana

Challenge

Install Filebeat and use the System Module
Install Filebeat with default settings and use the system module:
1. Download Filebeat:
```
	curl -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.2.3-x86_64.rpm
```
2. Install Filebeat:
```
	rpm --install filebeat-6.2.3-x86_64.rpm
```
3. Edit the system module to convert timestamp timezones to UTC:
  
  In /etc/filebeat/modules.d/system.yml.disabled, change:
```
	# Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
	#var.convert_timezone: false
```
  To:
```
	# Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
	var.convert_timezone: true
```
  For both the syslog and auth sections.
4. Enable the system Filebeat module:
```
	filebeat modules enable system
```
5. Install the ingest-geoip filter plugin for Elasticsearch ingest node:
```
	/usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-geoip
```
6. Restart Elasticsearch so it can use the new ingest-geoip plugin:
```
	systemctl restart elasticsearch
```
7. Once Elasticsearch starts up, push module assets to Elasticsearch and Kibana:
```
	filebeat setup
```
8. Enable and start Filebeat:
```
	systemctl enable filebeat
	systemctl start filebeat
```
Challenge

Connect to Kibana and Explore the Data
Connect to Kibana and explore your system log data:
1. From your local machine, SSH with port forwarding to your cloud server's public IP:
```
	ssh user_name@public_ip -L 5601:localhost:5601
```
2. Navigate to localhost:5601 in your web browser.
3. Go to the Dashboard plugin via the side navigation bar.
4. Search for system to filter to your system dashboards.
5. Explore your system log data with the supplied dashboards.

Author

A Cloud Guru

The Cloud Content team comprises subject matter experts hyper focused on services offered by the leading cloud vendors (AWS, GCP, and Azure), as well as cloud-related technologies such as Linux and DevOps. The team is thrilled to share their knowledge to help you build modern tech solutions from the ground up, secure and optimize your environments, and so much more!

What's a lab?

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.