Analyzing past System performance of a Linux server

Assumption: 

Today's date is 13th Aug, 2011.  You are asked to check the System performance of a Linux server on 7th Aug,2011 between 3 AM to 5 AM.

Solution: 

Run the 'sar' command on the respective 'sa' (System Activity) file created for the date "7th Aug,2011" with specifying the Starting and End time.

Illustration:

Go to /var/log/sa

[root@hostxyz sa]# ls -ltr sa??

-rw-r--r-- 1 root root 481776 Aug 5 23:50 sa05
-rw-r--r-- 1 root root 481776 Aug 6 23:50 sa06
-rw-r--r-- 1 root root 481776 Aug 7 23:50 sa07       # File that belongs to 7th Aug,2011
-rw-r--r-- 1 root root 481776 Aug 8 23:50 sa08
-rw-r--r-- 1 root root 481776 Aug 9 23:50 sa09
-rw-r--r-- 1 root root 481776 Aug 10 23:50 sa10
-rw-r--r-- 1 root root 481776 Aug 11 23:50 sa11
-rw-r--r-- 1 root root 481776 Aug 12 23:50 sa12
-rw-r--r-- 1 root root 287824 Aug 13 14:10 sa13

[root@hostxyz sa]#
[root@hostxyz sa]# sar -u -f /var/log/sa/sa07 -s 03:00:01 -e 05:00:01   # To check CPU utilization
Linux 2.6.18-92.el5 (hostxyz) 08/07/2011

03:00:01 AM CPU %user %nice %system %iowait %steal %idle
03:10:01 AM all 24.57 0.00 5.16 6.04 0.00 64.23
03:20:01 AM all 24.57 0.10 5.06 6.28 0.00 63.98
03:30:01 AM all 24.33 0.00 4.88 5.64 0.00 65.14
03:40:01 AM all 15.75 0.00 3.93 10.52 0.00 69.80
03:50:01 AM all 12.70 0.00 3.09 19.04 0.00 65.17
04:00:01 AM all 16.80 0.00 3.90 9.40 0.00 69.90
04:10:01 AM all 9.18 0.02 2.26 14.43 0.00 74.11
04:20:01 AM all 8.84 0.10 2.20 9.65 0.00 79.22
04:30:01 AM all 11.42 0.00 3.24 10.50 0.00 74.84
04:40:01 AM all 11.84 0.00 2.43 20.64 0.00 65.09
04:50:01 AM all 17.80 0.00 3.78 17.00 0.00 61.42
05:00:01 AM all 6.46 0.00 1.53 21.80 0.00 70.22
Average: all 15.35 0.02 3.46 12.58 0.00 68.59

[root@hostxyz sa]#

[root@hostxyz sa]#  sar -r -f /var/log/sa/sa07 -s 03:00:01 -e 05:00:01    # To check Memory status

[Output no shown]
.

[root@hostxyz sa]#  sar -q -f /var/log/sa/sa07 -s 03:00:01 -e 05:00:01    # To check Load average

[Output not shown]

.

[root@hostxyz sa]#  sar -b -f /var/log/sa/sa07 -s 03:00:01 -e 05:00:01     # To check I/O status

[Output not shown]

.

[root@hostxyz sa]#  sar -n DEV -f /var/log/sa/sa07 -s 03:00:01 -e 05:00:01    # To check Network status

[Output not shown]

.

[root@hostxyz sa]# 

Notes: In Linux, System activity report is collected for every 10 minutes by a cron job "sysstat" located under /etc/cron.d and at end of the day, a summary report is generated and saved in /var/log/sa/saXX file, which we can use for later analysis. 

[root@hostxyz cron.d]# cat sysstat

# run system activity accounting tool every 10 minutes

*/10 * * * * root /usr/lib64/sa/sa1 1 1

# generate a daily summary of process accounting at 23:53
53 23 * * * root /usr/lib64/sa/sa2 -A

root@hostxyz cron.d]#