SAR COMMAND IN LINUX

 SAR COMMAND IN LINUX

SAR is very much helpful in doing exactly that.

The first thing that we need to do is check and confirm whether you have SAR utility installed on the machine. Which can be checked by listing all rpm's and finding for this utility.

SAR is one of the utility inside sysstat. You can easily download and install it in your machine very easily through YUM. (But yeah dont worry because most of the distribution comes prepacked with sysstat tool).

[root@ ~]# yum install sysstat

SAR (System Activity Reporter) will Give Information about the following things:

1. System Buffer activity
2. Information about system calls
3. Block device information
4. Overall paging information
5. Semaphore and memory allocation information
6. CPU utilization and process report

The main thing that we need to understand regarding SAR is that, everything is done using a cron. By default in many Linux distribution you will have a file named /etc/cron.d/sysstat.

Lets see how really SAR works.

If we start thinking about system monitoring, then the tool must have each and every data about the system's different aspects and must cover all time intervals. Which means a monitoring system must be able to provide the statistics of the machine for a given time.

There is no way, other than taking all the metrics and statistics of the machine at a definite time interval. Reducing the time interval for collecting the statistics will increase the amount of detailed statistics we have(because we will be having more data about the system).

SAR does exactly that. sar takes the statistics of different aspects of the machine at a definite time interval. So SAR runs through CRON.

[root@myvm ~]# cat /etc/cron.d/sysstat

# run system activity accounting tool every 10 minutes

*/10 * * * * root /usr/lib64/sa/sa1 1 1

# generate a daily summary of process accounting at 23:53

53 23 * * * root /usr/lib64/sa/sa2 -A

• So it can be seen from the above cron file for SAR that its running "sa1" script located at "/usr/lib64/sa/" at every 10 minutes
• And is also running a script /usr/lib64/sa/sa2 at the end of the day at around 23.53

So the first cron entry for SAR(/usr/lib64/sa/sa1) will run every 10 minutes which inturn will call the sadc utility to collect system stats and store it in a binary file (one file for a day)

And the second cron entry will dump all the contents of that binary file into another text file, and purges data older than a particular number of days, Normally 7 days by default(which is mentioned in the following file),

[root@archive ~]# cat /etc/sysconfig/sysstat

# How long to keep log files (days), maximum is a month

HISTORY=7

So you can modify that HISTORY entry easily by editing the file.

So although the system statistics is being collected every 10 minutes through cron(modify the cron to run every 1 minute for more accurate information) If you want to see the stats, then you need to run the command as below.

The simple sar command output is as shown below.

12:00:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle

12:01:01 AM     all     73.28      0.00      1.25      0.00      0.00     25.47

12:02:01 AM     all      7.83      0.00      0.44      0.00      0.00     91.73

It can be seen from the output that its reporting me the output of the collected stats for every minute(which means i have my cron at 1 minute interval), and will show the details of the whole day(or will show details collected till when you typed the command).

Understanding the output of SAR command

%user: This shows the total time that the processor is spending on different process YCX5UKN5ZKEJ

%sys: this shows the percentage of time spend by the processor for operating system tasks(because the previous user shows the time spend for user end process)

%iowait: the name iowait itself suggests that its the time spend by processor waiting for devices(input and output)

%nice: Most of you guys must be knowing that a user can change the priority of a process in linux by changing the nice value in Linux. This table shows the time spend by CPU for process whose nice value has been changed.

%steal:  This column shows the amount to time spend by a CPU (which is virtualized), for resources from the physical CPU

%idle: This suggests the idle time spend by the processor.

By default sar stores all of its data under /var/log/sa/ and a days are named as shown below.

s01 - for first day of the month

s02-for second day of the month

s03,s04..and so on.

-d option in SAR command

This -d option can be used to report each and every activity related to different devices attached to the system(block devices). A typical output of the sar command with -d option is shown below.

12:00:01 AM       DEV       tps  rd_sec/s  wr_sec/s  avgrq-sz  avgqu-sz     await     svctm     %util

12:01:01 AM    dev3-0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

12:01:01 AM   dev3-64      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00

12:01:01 AM    dev8-0     55.62      9.98   8317.87    149.72     13.68    245.96      2.61     14.52

12:01:01 AM   dev8-16      0.00      0.00      0.00      0.00      0.00     

DEV: this column names devices on the machine, according to major and minor numbers of a Linux block device. You can check this by doing an ls -l in /dev directory. as shown below.

brw-r-----  1 root disk   8,    0 Nov 16 16:29 sda

brw-r-----  1 root disk   8,    1 Nov 16 16:29 sda1

brw-r-----  1 root disk   8,    2 Nov 16 16:29 sda2

in the above shown "ls -l" outut for "sda", major number is "8",and minor number is "0"...So you can easily identify the disk mentioned in the sar command output.

tps: tps stands for transfer per second, so it shows the transfer per second to that particular device

rd_sec/s: this shows you the total number of sectors on that device which is being read

wr_sec/s: if rd_sec/s is sectors being read per second then obviously wr_sec is sectors being written per second.

avgrq-sz: this column shows the average.

await: this shows the total number of time that the processor waited for requests regarding IO

%util: this column shows the usage of cpu in percentage when the request was generated

Show Memory usage in SAR command

the -r option available in sar command is very much useful. it shows the memory,swap,cached memory etc at every interval or required time interval.

02:20:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached kbswpfree kbswpused  %swpused  kbswpcad

02:30:01 AM    609500   1487652     70.94    242420    777560   1075980       364      0.03       360

02:40:01 AM    609500   1487652     70.94    242424    777568   1075980       364      0.03       360

in the above output most of the columns are self explanatory(and most of the outputs are in KB).

kbmemfree: this shows the amount of free memory

Kbmemused: memory used

%memused: percentage of memory used

kbbuffers: buffer memory used by the kernel.

kbcached: cached memory used by the kernel

all other entries for memory are swap(free,used,percentage etc)

How to fetch metrics of a particular day using SAR in linux

As mentioned before all the metrics for a particular day are saved in sa wise. So if i want to know my metrics for 27 th day the month i can easily find that out as shown below.

[root@archive ~]# sar -f /var/log/sa/sa27

Linux 2.6.18-194.el5xen (archive.r)     11/27/2012

02:20:01 AM       CPU     %user     %nice   %system   %iowait    %steal     %idle

02:30:01 AM       all      2.58      0.00      0.70      1.12      0.05     95.55

02:40:01 AM       all      2.56      0.00      0.69      1.05      0.04     95.66

in the above command we have passed /var/log/sa/sa27 as an argument as i needed stats for that day..pass the saas you require in the above command

How to fetch SAR metrics for a specific time on a particular date

this can be achieved by passing another argument as shown below.

[root@archive ~]# sar -f /var/log/sa/sa27 -s 02:20:00 -e 03:20:00

Linux 2.6.18-194.el5xen (archive.r)     11/27/2012

02:20:01 AM       CPU     %user     %nice   %system   %iowait    %steal     %idle

02:30:01 AM       all      2.58      0.00      0.70      1.12      0.05     95.55

02:40:01 AM       all      2.56      0.00      0.69      1.05      0.04     95.66

in the above shown example i asked sar to fetch the metrics between 2:20:00 and 3:20:00 on 27th day of the month

You can even pass any other metric option along with the time interval...such as -d or -r

Using -A option along with the above command will show ALL (all the metrics collected by sar).

sar -f /var/log/sa/sa27 -s 02:20:00 -e 03:20:00 -A

The output will be elaborate. and you will get almost everything in sar from that -A option on your screen!

Show network statistics using sar command

sar command even shows network statistics. This can be done by using the -n DEV option in sar command.

[root@archive ~]# sar -n DEV

Linux 2.6.18-194.el5xen (archive.r)     11/27/2012

02:20:01 AM     IFACE   rxpck/s   txpck/s   rxbyt/s   txbyt/s   rxcmp/s   txcmp/s  rxmcst/s

02:30:01 AM        lo      0.01      0.01      0.77      0.77      0.00      0.00      0.00

02:30:01 AM      eth0     12.30      0.12   1285.06     27.59      0.00      0.00      0.00

IFACE: stands for the nic card interface name

rxpck/s: this shows the total packets received per second

txpck/s:transmitted packets per second

rxcmp/s: compressed packets received

txcmp/s: compressed packets transmitted

rxmcst/s: packets multicasted per second.

Some other Metrics that can be determined using sar

-y option in sar:  can be used to determine tty details

-X option in sar to get details of a particular process. You need to pass pid as an argument to this option.

-n SOCK option in sar: this option will report all socket details.

Telnet and Curl to check the HTTP content.

Telnet and Curl to check the HTTP content.

Example 1.
=========================================

telnet: > telnet www.example.com http
telnet: Trying 192.0.2.2...
telnet: Connected to www.example.com.
telnet: Escape character is '^]'.
client: HEAD /example/example.shtml HTTP/1.1
client: Host: www.example.com
client: Connection: close
client: 
server: HTTP/1.1 200 OK
server: Date: Wed, 21 Jan 2004 22:13:05 GMT
server: Server: Apache/1.3.12-Turbo
server: Connection: close
server: Content-Type: text/html 

=========================================

Example 2.

 # telnet 171.189.222.30 30651
Trying 171.189.222.30...
Connected to 171.189.222.30.
Escape character is '^]'.
GET /monitor/bigip.html HTTP/1.0
Host: uat.example.com

HTTP/1.1 200 OK
Server: Sun-ONE-Web-Server/6.1
Date: Mon, 24 Jun 2013 02:46:18 GMT
Content-length: 9
Content-type: text/html
Last-modified: Wed, 19 Aug 2009 15:08:24 GMT
Accept-ranges: bytes
Connection: close

ServerUp
Connection closed by foreign host. =========================================

Curl Example.

curl -v --header "Host: uat.example.com" --header "User-Agent: BigIP Prober" http://171.189.222.29:30651/monitor/bigip.html
curl -v --header "Host: uat.example.com" --header "User-Agent: BigIP Prober" http://171.189.222.30:30651/monitor/bigip.html

=========================================

SSL CERTIFICATE CHAIN

SCRIPT TO VIEW BIGIP LTM V10 VIRTUAL SERVER CONFIGURATION.

SCRIPT TO VIEW BIGIP LTM V10 VIRTUAL SERVER CONFIGURATION 

#!/bin/bash
> out.txt
> logcheck.txt
> result.txt
read -p "Enter Username: " user
read -p "Enter Valid Password: " password
if [ -z "${user}" ] || [ -z "${password}" ];   # Exit if no arguments were given.; then
then
echo "Error: Username or Password are empty"
exit $NOARGS
fi ;
read -p "Enter Active Device name: " dev
if [ -z "${dev}" ] ;   # Exit if no arguments were given.
then
echo "Error: Device Name is empty"
exit $NOARGS
fi ;
read -p "Enter Valid VIP IP: " vipip
if [ -z "${vipip}" ] ;   # Exit if no arguments were given.
then
echo "Error: VIP IP is empty"
exit $NOARGS
else
/usr/local/bin/sshpass -p "$password" ssh  "$user"@"$dev" "cat /config/bigip.conf |grep -q $vipip"
errorCode=$?
if [ ${errorCode} -ne 0 ]
then echo "Unable to find the IP in device config.Please confim."
exit $NOARGS
fi ;
fi ;

read -p "Enter Valid VIP Port: " vipipport
echo "$user"
echo "$dev"
echo "$password"
/usr/local/bin/sshpass -p "$password" ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no "$user"@"$dev" "exit"
errorCode=$?    # checks if the last operation (sqlplus) was completed successfully or not
if [ ${errorCode} -ne 0 ]
then
echo "Unable to connect the Device with the supplied credentials. Please check the Credentials or Device Name."
exit ${CRED_ERR}
else
echo  "Device was connected successfully"
fi
/usr/local/bin/sshpass -p "$password" ssh  "$user"@"$dev" "cat /config/bigip.conf |grep $vipip:$vipipport -A 2 -B 4| egrep 'virtual|pool' | awk '{print "'$1,$2'"}'" >> out.txt
IFS=$'\n';for pool in `cat out.txt |grep pool` ; do /usr/local/bin/sshpass -p "$password" ssh "$user"@"$dev" " b $pool list |grep 'monitor'|uniq | sed 's/\//g'|awk '{print "'$1,$2'"}'" >> out.txt ;done
IFS=$'\n'; for line in `cat out.txt|awk '!x[$0]++'`; do echo "===================================" && /usr/local/bin/sshpass -p "$password" ssh "$user"@"$dev" " b $line list" ; done
FS=$'\n'; for line in `cat out.txt|awk '!x[$0]++'`; do echo "===================================" && /usr/local/bin/sshpass -p "$password" ssh "$user"@"$dev" " b $line show" ; done > result.txt 2>&1

read -p "Do You Want To check The Logs? [y/n]" answer
if [[ $answer =~ ^(yes|y|Y| ) ]]
then
cat out.txt|awk '!x[$0]++'|egrep 'virtual|pool'|awk {'print $2'} >> logcheck.txt
cat result.txt|grep "POOL MEMBER"|awk {'print $4'}|cut -f2 -d"/"|cut -f1 -d ":"|awk '!x[$0]++' >> logcheck.txt
searchvalue=`cat logcheck.txt |xargs| sed 's/ /|/g'`
echo "search value $searchvalue"
/usr/local/bin/sshpass -p "$password" ssh  "$user"@"$dev" "cat /var/log/ltm && zcat /var/log/ltm.* |egrep -w '$searchvalue'"
else
exit
fi

Samba 4 - Active Directory Domain Controller in centos 6.2.

Samba 4 - Active Directory Domain Controller in Centos 6.2

=================================

Hostname: server.mycwwtest.com 

IP Address: 172.16.23.43.

DNS: 172.16.23.43.
=================================

Installation of Samba V. 4.5.
-----------------------------

Query your rpm database to find any instances of older samba packages:

# rpm -qa | grep samba

We have to change the hostname:

#vi /etc/sysconfig/network

HOSTNAME= server.mycwwtest.com

Add the hostname in hosts.

#vi /etc/hosts

172.16.23.43 server.mycwwtest.com server

Edit resolv.conf:

# vi /etc/resolv.conf

nameserver 172.16.23.43

search mycwwtest.com

We are not using selinux hence Disable selinux:

# vi /etc/sysconfig/selinux

 SELINUX=disabled

# setenforce 0

If there are any older samba packages remove them with YUM:

# yum remove samba-winbind-client samba-common samba-client

Install the following packages required for installing and building Samba 4:

#yum install glibc glibc-devel gcc python* libacl-devel krb5-workstation krb5-libs

pam_krb5 git gcc libacl-devel libblkid-devel gnutls-devel  readline-devel

python-devel gdb pkgconfig zlib-devel setroubleshoot-server  

setroubleshoot-plugins policycoreutils-python libsemanage-python 

setools-libs-python setools-libs popt-devel libpcap-devel sqlite-devel 

libidn-devel libxml2-devel libacl-devel libsepol-devel libattr-devel 

keyutils-libs-devel cyrus-sasl-devel make wget python-devel gnutls-devel 

openssl-devel libacl-devel krb5-server krb5-libs krb5-workstation bind 

bind-libs bind-utils

We are going to use internal Kerberos and bind so we have to stop the services.

#/etc/init.d/kadmin stop

#/etc/init.d/named stop

#/etc/init.d/krb5kdc stop

#chkconfig kadmin off

#chkconfig named off

#chkconfig krb5kdc off

You can download any stable version or checkout the current version from git repository of samba(git://git.samba.org/samba.git samba-master).
Now I am using a stable version.

# wget http://ftp.samba.org/pub/samba/samba-4.0.5.tar.gz

# tar -xzvf samba-4.0.5.tar.gz

# cd samba-4.0.5/

# ./configure --enable-selftest

# make && make install

You will have to add the PATH variable for samba4 in your machine.

# export PATH=$PATH:/usr/local/samba/sbin:/usr/local/samba/bin

Provision Samba 4

The provision step sets up a basic user database, and is used when you are setting up your Samba4 server in its own domain.

# /usr/local/samba/bin/samba-tool domain provision –-use-rfc2307 --interactive

The 'domain provision' tool should pick defaults for you automatically. Change to your configurations if necessary:

Realm [MYCWWTEST.COM]:

Domain [MYCWWTEST]: (press Enter)

Server Role (dc, member, standalone) [dc]: (press Enter)

DNS backend (SAMBA_INTERNAL, BIND9_FLATFILE, BIND9_DLZ, NONE) [SAMBA_INTERNAL]: (press Enter)

DNS forwarder IP address (write 'none' to disable forwarding) [172.16.23.43]:8.8.8.8

Administrator password: 

Retype password:

Edit named configuration:

# vi /etc/named.conf

options {

listen-on port 53 { any; };

forwarders {192.168.1.8; };

allow-query { any; };

tkey-gssapi-keytab "/usr/local/samba/private/dns.keytab";  ==(add this line)

};

include "/usr/local/samba/private/named.conf"; ==(add this line)

Configure Kerberos

In CentOS 6.3 or 6.4, kerberos is handled by the '/etc/krb5.conf' file. Make a backup copy of this original file, and then replace the existing file, if any, with the sample from /usr/local/samba/share/setup/krb5.conf.

cp /usr/local/samba/share/setup/krb5.conf /etc/krb5.conf

Start Samba 4 AD DC

Start the samba daemon:

# /usr/local/samba/sbin/samba

If you would like Samba to start at boot, append the following to your

'/etc/rc.d/rc.local' file:

/usr/local/samba/sbin/samba

Testing Samba as an Active Directory DC

Verify you are indeed running the correct version of Samba. Your version should start with version 4 (note: samba daemon must be running):

# /usr/local/samba/sbin/samba -V

Version 4.0.5

Verify you are running the correct samba-client version:

# /usr/local/samba/bin/smbclient --version

Version 4.0.5

Now run this command to list the shares on your Samba4 server:

# /usr/local/samba/bin/smbclient -L localhost -U%

Domain=[MYCWWTEST] OS=[Unix] Server=[Samba 4.0.5] 

        Sharename       Type      Comment 

        ---------       ----      ------- 

        netlogon        Disk      

        sysvol          Disk      

        IPC$            IPC       IPC Service (Samba 4.0.5) 

Domain=[MYCWWTEST] OS=[Unix] Server=[Samba 4.0.5] 

        Server               Comment 

        ---------            ------- 

        Workgroup            Master 

        ---------            -------

Testing ports

# netstat -tapn | grep samba (check port 53/88/389/636 are listening by samba)

tcp    0    0    0.0.0.0:53      0.0.0.0:*    LISTEN    26182/samba

tcp    0    0    0.0.0.0:88      0.0.0.0:*    LISTEN    26182/samba

tcp    0    0    0.0.0.0:389     0.0.0.0:*    LISTEN    26182/samba

tcp    0    0    0.0.0.0:636     0.0.0.0:*    LISTEN    26182/samba

Testing Kerberos

The simplest test is to use the 'kinit' command as follows:

# kinit administrator@MYCWWTEST.COM

Password for administrator@MYDOMAIN.COM:

Warning: Your password will expire in 41 days on Sun Feb 3 14:21:51 2013

'kinit' will not give you any output. To verify that Kerberos is working, and that you received a ticket, run the following:

# klist -e

Ticket cache: FILE:/tmp/krb5cc_0

Default principal: administrator@ MYCWWTEST.COM

Valid starting Expires Service principal

12/23/12 15:39:28 12/24/12 01:39:28 krbtgt/ MYCWWTEST.COM@MYCWWTEST.COM

renew until 12/24/12 15:39:19

Testing DNS

Make sure that samba is running and then test to make sure that DNS is working properly. Run the following commands and compare the output to what is shown:

#dig server.mycwwtest.com

#dig mycwwtest.com

#dig –x 172.16.23.43

# host -t SRV _ldap._tcp.mycwwtest.com.

_ldap._tcp. mycwwtest.com has SRV record 0 100 389 server.mycwwtest.com.

# host -t SRV _kerberos._udp.mycwwtest.com.

_kerberos._udp. mycwwtest.com has SRV record 0 100 88 server.mycwwtest.com.

# host -t A server.mycwwtest.com.

server.mycwwtest.com.com has address 172.16.23.43

#nslookup server.mycwwtest.com 172.16.23.43

#nslookup –type=srv _kerberos._udp.mycwwtest.com.

#nslookup –type=srv _kerberos._tcp.mycwwtest.com.

 

NTP (Network Time Protocol)

Make sure that 'ntpd' is running and installed. If 'ntpd' is not installed you can install it with YUM:

# yum install ntp

Configuring NTP:

# vi /etc/ntp.conf

ntpsigndsocket /usr/local/samba/var/lib/ntp_signd/

restrict default kod nomodify notrap nopeer mssntp

Enable ntpd:

# /etc/init.d/ntpd start

Also, use the 'chkconfig' command to have ntpd run at boot:

# chkconfig ntpd on

Configure Windows Client to Join Domain

The following will describe how to add a Windows 7 client to the samba DC. For other versions of Windows the same principle should be the same.

To simplify and to limit errors with DHCP, we will assign a static IP address to our Windows 7 client NIC. Configure your network device as follows:

Click 'OK' to save the changes.

Now bring up a command prompt in windows and ping the Samba DC:

ping 172.16.23.43

Verify that DNS is working correctly by pinging the FQDN:

ping server.mycwwtest.com

run è cmd è nslookup è server.mycwwtest.com

run è cmd è nslookup è mycwwtest.com

run è cmd è nslookup è 172.16.23.43

If you get replies from both then this is a good sign and should mean that your Samba DC is functional. Also, you may need to reboot Windows for network settings to take effect.

Join Windows 7 Client to the Domain

1. Right-click 'My Computer' icon and choose 'Properties'

2. From the left-side pane click 'Advanced system settings'

3. Choose the 'Computer Name' tab and click 'Change...'

4. Select option 'Domain', and insert MYCWWTEST.COM. If this fails just try MYCWWTEST.

 

5. When it requests a username and password, type 'Administrator' as the username and then enter your password. (password = the password you used when you ran the 'samba-tool domain provision' command)

6. You should get a message box stating 'Welcome to the MYCWWTEST.COM domain'

7. Click OK on this message box and the Properties window, and you will then be instructed to restart your computer.

8. After restarting you should be presented with the normal login dialog. Click on 'Switch User' button.

9. Choose 'Other user' and then enter in the following:

 Press 'Enter' or the arrow button.

10. You should then authenticate and then login to Windows

Install Windows Remote Administration Tools

To install the GUI tools to manage the domain you must install the Remote Server Administration Tools. This will allow you easily manage the domain using Active Directory.

Windows 7

1. Download the Windows Remote Server Administration Tools

2. Follow the 'Install RSAT' instructions

3. Enable the necessary components in 'Control Panel -> Programs -> Turn Windows features on or off -> Remote Server Administration Tools'

4. You may need to add the Administrative Tools to your start menu. Right-click 'Start button' and select 'Properties -> Start Menu tab - Customize... -> System administrative tools - Display on the All Programs menu'

Configure roaming profile in samba 4 on centos 6.2

A roaming profile is a feature of Domain server which allows users to access their personal data on any computer which is joined under the same domain on the same network. So basically the users are logging into their profile whose location is defined on the domain server. So every time a user logs in they are redirected to their profile location so this action becomes computer independent.

#mkdir  /usr/local/samba/var/profiles

Now make an entry of this directory in smb.conf to make it sharable among the domain users

# vi /usr/local/samba/etc/smb.conf

   [profiles]
       path = /usr/local/samba/var/profiles
       read only = no

# /etc/init.d/samba4 restart

# killall samba

# kinit administrator@MYCWWTEST.COM

Once your samba is restarted you can check the updates shared directory on your server

# smbclient -L localhost -U%

        Sharename       Type       Comment

        ---------       ----       -------

        netlogon        Disk

        sysvol          Disk

        profiles        Disk

        IPC$            IPC        IPC Service

Now go to your windows client machine where you have installed the remote administration tool for your domain server. 

Go to run  dsa.msc

Go to the user for whom you want to implement roaming profile and add the path followed by the username of the user of the profile directory under the profile tab of the properties section as shown below :

Now run this command to update the policies on the users.

Go to your server and you can find a directory by the name of the user must have been created under profile directory

 # cd /usr/local/samba/var/profiles

 # ls -l

 drwxr-xr-x 32 450012 users   4096 Jan  2 15:35 user1