Argument list too long error

At times, you might get into a situation where you can't copy, move, remove or even list a huge number of files in a directory. When you attempt it, an error will be thrown saying "Argument list too long". This actually indicates that the OS has exceeded its Argument list limit. To know the Argument list limitation of your OS, run the following command "getconf ARG_MAX". It will show you the maximum number of arguments that can be passed to mv, cp, rm or ls command.

# getconf ARG_MAX
   131072

To get rid of this problem, you must split your mv or cp command using wildcards.

Like instead of using "cp /folder1/* /folder2/ ",  we should use "cp /folder1/a* /folder2/; cp /folder1/b* /folder2/"
(or)
you can use 'for' loop construct as follows:

# for i in {a..z}; do cp /folder1/$i* /folder2/ ; done &
Please note in 'for' loop, for every cp command it executes, it creates a new process id.

Oracle ASM commands

Some Oracle ASM admin commands for my records :

oracleasm status
oracleasm listdisks
oracleasm start
oracleasm stop
oracleasm querydisk
oracleasm querydisk /dev/sd*
ls -l /dev/oracleasm/disks
oracleasm listdisks | xargs oracleasm querydisk -p
oracleasm scandisks
oracleasm scandisks -p
oracleasm restart
oracleasm enable
oracleasm disable
oracleasm configure
oracleasm createdisk ASMDISKName /dev/sdx
/usr/sbin/asmtool -C -l /dev/oracleasm -n ASMDISKName -s /dev/sdx -a force=yes
oracleasm deletedisk

Creating complex password

There could be multiple ways to create Complex password using Linux OS. Here are couple of simple ways with sample execution to create complex password of length 8 characters:

linuxhost $ openssl rand -base64 6
NRwijHK9
linuxhost $ < /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c8; echo
7aSNmEot
linuxhost $

Scripted version, where you can define the characters that you require in the password:
http://smart-scripts-for-sysadmins.blogspot.com/2011/03/script-for-generating-complex-password.html

Listing history commands using 'fc'

In Linux, the command 'fc' (stands for find command) can be used as an alternative to 'history'. 

This is a Bash built-in command using which we can list only the selected number of last executed commands.

For example, if you wish to list only last 20 commands, the syntax for 'fc' command should be "fc -l -20".

Below shown is a sample execution:

linuxhost # fc -l -20

1081     ls

1082     cd current/

1083     ls

1084     make

1085     make linux

1086     make target

1087     history | grep iozone

1088     iozone -a -i 0 -i 1 -i 2 -r 32k

1089     df -h

1090     ps -fu gpadmin | grep iozone

1091     man iozone

1092     exit

1093     df -h

1094     df -h

1095     cd /gplum/

1096     du -sh .

1097     top

1098     uname -r

1099     free -m

1100     getconf -a | grep -i pagesize

linuxhost #

Finding all the binaries in a Linux machine

# find / -xdev -type f -perm +111 -exec file -i '{ }' \; | grep 'x-executable' > list_of_binaries.txt

# awk -F":" '{print $1}' list_of_binaries.txt > List_of_Binaries.txt

Comparing directories between 2 Linux systems

This is a continuation of my previous post on comparing directories using 'diff' command that are Local.
However on several occasions I faced situations, where I need to compare the file configuration between 2 Linux machines (especially in Application servers). In those scenario what I normally used to do was, copy the entire directory from server to another and run the 'diff' command as shown in the previous post. It was obviously a tedious task. Eventually I found a solution to simplify this task by using 'sshfs', which is Secure SHell FileSystem (refer 'man sshfs' for more details).

Steps to compare Folders in 2 machines. Let's assume the local folder name "/dir1" and the remote folder name is "remotehost:/dir2":

1. Login to first machine as 'root' user.

2. Create a local mount point to mount the remote folder.
    # mkdir /rmtmnt

3. Mount the Folder in remote machine using 'sshfs'
    # sshfs user@remotehost:/dir2  /rmtmnt

4. Run the 'diff' command
     # diff /dirl /rmtmnt -r --brief

5. After the execution, umount the remote filesystem using 'fusermount -u'.
     # fusermount -u /rmtmnt
    
Please note 'umount' command wouldn't work with sshfs.

Comparing directories using 'diff' command

'diff' command can be used to compare not just Files but also Directories/Folders as well.
Following is a self-explanatory demonstration on how the 'diff' command can used to compare or find differences between 2 folders (folder1 & folder2).

[root@hostxyz folder1]# ls -l
total 16
-rw-r--r-- 1 root root 3 Sep 24 12:29 oraclelinux
-rw-r--r-- 1 root root 3 Sep 24 12:28 redhat
-rw-r--r-- 1 root root 3 Sep 24 12:28 suse
-rw-r--r-- 1 root root 3 Sep 24 12:29 ubuntu
[root@hostxyz folder1]# for i in `ls`; do echo $i   : `cat $i`; done      # Contents of all files in folder1
oraclelinux : OS
redhat : OS
suse : OS
ubuntu : OS
[root@hostxyz folder1]# cd ../folder2
[root@hostxyz folder2]# ls -l
total 16
-rw-r--r-- 1 root root 17 Sep 24 12:34 CentOS
-rw-r--r-- 1 root root 17 Sep 24 12:30 oraclelinux
-rw-r--r-- 1 root root  3 Sep 24 12:29 redhat
-rw-r--r-- 1 root root 17 Sep 24 12:30 ubuntu
[root@hostxyz folder2]# for i in `ls`; do echo $i   : `cat $i`; done     # Contents of all files in folder2
CentOS : Operatins system
oraclelinux : Operating system
redhat : OS
ubuntu : Operating system

[root@hostxyz folder2]# diff /var/folder1 /var/folder2 -r --brief     
Only in /var/folder2: CentOS
Files /var/folder1/oraclelinux and /var/folder2/oraclelinux differ
Only in /var/folder1: suse
Files /var/folder1/ubuntu and /var/folder2/ubuntu differ
[root@hostxyz folder2]#

Options used :
 -r  :      Searched recursively through the directories.
--brief : Only shows the names of the files that differ. If you want details of the content that differs, remove this option.

Finding the device-name associated with a file-system label

We might have came across a situation where a Device associated with a file-system label is missing and due to which we will get an error on boot-up saying file-system can't be found for the Label=[labelname]. To resolve this, we have to find the corresponding Device name associated with the file-system label (set by using the command 'e2label') and fix the problem with it by running 'fsck' or by changing the Label to other alternative disk.
'findfs' is the handy command for it; this is like a reverse case of 'e2label' command.

Below given is a self-explanatory illustration of it:

[root@linuxhost ~]# e2label /dev/sda1
/boot
[root@linuxhost ~]# e2label /dev/sda1 Test
[root@linuxhost ~]# e2label /dev/sda1
Test

[root@linuxhost ~]# findfs LABEL="Test"
/dev/sda1
[root@linuxhost ~]# 

[root@linuxhost ~]# cd /dev/disk/by-label/
[root@linuxhost ~]# ls -l
total 0
lrwxrwxrwx 1 root root 10 Aug 16 08:56 Test -> ../../sda1
[root@linuxhost ~]#

Total number of 'D' processes

Executing this command-set will display you the total number of uninterruptable sleep (D) processes on the server:

# ps aux | awk '{if ($8 == "D") {print; count++} } END {print "Total No. of uninterruptable Sleep Processes: "count}'

Checking the progress of 'dd' command execution

Start the 'dd' command in back-ground:
[root@hostxyz ~]# dd if=/dev/sda of=/dev/sandisk1 &
[1] <PID>

Execute either of  following while statement against the PID captured in the previous command:
[root@hostxyz ~] # while true; do kill -10 <PID>; sleep 10; clear; done
(or)
root@hostxyz ~] # while true; do kill -USR1 <PID>; sleep 10; clear; done

SAMPLE EXECUTION

[root@hostxyz ~]# dd if=/dev/zero of=/dev/sdf1 &
[1] 2069
[root@hostxyz ~]# while true; do kill -10 2069; sleep 5; clear; done
14207400+0 records in
14207400+0 records out
7274188800 bytes (7.3 GB) copied, 38.1668 s, 191 MB/s
15276232+0 records in
15276232+0 records out

7821430784 bytes (7.8 GB) copied, 43.1825 s, 181 MB/s

17829523+0 records in

17829523+0 records out

9128715776 bytes (9.1 GB) copied, 48.1945 s, 189 MB/s

18739498+0 records in

18739498+0 records out
.
.
<so on>

Environment variables in Linux

On a Linux system, we can view the Environment variables that are set and exported using 3 built-in commands: 

'set', 'env' & 'export -p'.

Ever wondered how can we see the other 'unset' environment variables belongs to Bash shell or some command or tool ?

The technique that I follow is to look in 2 places: One in the Binaries of the program which the application uses and other in the MAN page.

To view all the printable strings in a Bash shell:  # strings /bin/bash | grep -P '[A-Z]+'

In this output, there will be a lot of additional stuff and you will have to look for environment variables carefully

To print all the ENV variables related to 'history' command:

# man bash | sed 's/[[:cntrl:]].//g' | egrep -x ' +HIST[[:alpha:]]*'

       HISTCMD

       HISTCONTROL

       HISTFILE

       HISTFILESIZE

       HISTIGNORE

       HISTSIZE

       HISTTIMEFORMAT

A URL that contains all the Bash variables explaining each:

http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_03_02.html

Improving Read performance of Disks using 'blockdev'

The Read performance of a Disk can be improved by increasing a parameter called "Read+Ahead" using 'blockdev' command. By default the Linux OS will read 128 KB of data in advance so that it is already in Memory cache before the program needs it. This value can be increased so as to get better Read Performance.

Steps to check and increase the Read-Ahead value:

To check the current 'blockdev' status of all block device:   # blockdev --report
[root@linuxserver ~]# blockdev --report
RO    RA   SSZ   BSZ   StartSec     Size    Device
rw 16384   512  4096          0   14680064  /dev/sda
rw 16384   512  1024         63     208782  /dev/sda1
rw 16384   512   512     208845   10265535  /dev/sda2
rw 16384   512  4096          0 2147483648  /dev/sdb
rw 16384   512  4096          0   35651584  /dev/sdc
rw 16384   512  2048         63   35648172  /dev/sdc1
rw 16384   512  4096          0  104857600  /dev/sdd

To check the 'Read-Ahead' value of an individual disk (let's take sda)
# blockdev --getra /dev/sda
(or)
# cat /sys/block/sda/queue/read_ahead_kb

To change the 'Read-Ahead' value to 8 MB (16384 times of 512 bytes blocks).
# blockdev --setra 16384 /dev/sda

To make it permanent upon system reboot, just add this command entry in /etc/rc.local.

Tweaking 'ps' command to display full user-name

By default, the UID column in ‘ps’ output has a character length of 8. If the user name exceeds 8 characters, it will convert the User-name to its corresponding UID and displays it. This might cause inconvenience when we want to check the process list run by a user-account with more than 8 characters. 'ps' command has an option to display full user-name.

Normal 'ps' command:

[longusername@hostxyz ~]$ id longusername
uid=55062(longusername) gid=55062(longusername) groups=55062(longusername)
[longusername@ora-prod-inf-d2 ~]$ ps -ef | grep tail
55062    11708 11470  0 02:11 pts/2    00:00:00 tailf a
55062    11739 11470  0 02:11 pts/2    00:00:00 tailf b
55062    11754 11470  0 02:11 pts/2    00:00:00 tailf c
55062    20637 11470  0 02:33 pts/2    00:00:00 grep tail

A tweaked ‘ps’ output with User column set as 20 characters 

[longusername@hostxyz ~]$ ps -o user:20,pid,ppid,c,stime,tty,cmd | grep tail
longusername         11708 11470  0 02:11 pts/2    tailf a
longusername         11739 11470  0 02:11 pts/2    tailf b
longusername         11754 11470  0 02:11 pts/2    tailf c
longusername         20641 11470  0 02:33 pts/2    grep tail
[longusername@hostxyz ~]$

In this you can see the full user-name getting displayed.

Additional command :

List out the processes based on CPU and Memory consumption

# ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head

Try 'compgen' - an interesting command

'compgen' is a handy Bash built-in command to list all the Commands, Aliases and Bash Built-ins functions available for a logged-in user.
To list all the commands     :  compgen -c
To list all the Aliases          :   compgen -a
To list all the Bash Built-ins :  compgen -b

How to check when a patch was last installed or upgraded ?

Syntax: rpm -qa --last | grep <package name>

Example:  Suppose you want to know when the MySql packages was last installed or upgraded.

The syntax should be:  
[root@Hostxyz ~]# rpm -qa --last | grep -i mysql
MySQL-client-5.6.5_m8-1.rhel5                 Fri 13 Jul 2012 02:35:01 AM PDT
MySQL-server-5.6.5_m8-1.rhel5                Fri 13 Jul 2012 02:34:44 AM PDT
php54-mysql-5.4.0-1.el5                           Fri 13 Jul 2012 01:17:21 AM PDT
[root@Hostxyz ~]#

To list the entire package list, just execute # rpm -qa --last

Complex Arithmetic using Linux Binary calculator

Lets say you want to execute the below arithmetic expression using Binary calculator (bc):
((10*10)+100)/4. The expected value for this arithmetic expression is 50.

The syntax using 'bc' would be :
# echo "10*10;.+100;./4" | bc
100
200
50

#

Please note the semicolon(;) and period (.) used in-between.

Other version using 'last' keyword:

# echo "10*10;last+100;last/4" | bc

100

200

50

#

The period (.) or 'last' keyword denotes result of previous execution.

Rsync command syntax

Below given is the recommended Rsync syntax which can be used to sync 2 file-systems. However appropriate options can be chosen based on the requirement:
 Source machine# nohup rsync -atlrzvuop --progress /<source folder>/  <Destination IP/Hostname>:/<Dest folder>/  &
-a  :  Archive mode
-t  :   Preserve modification time
-l  :   Preserve symbolic links
-r :   Copy recursively
-v :   Verbose
-z  :  Zip the files before transfer
-u :  Skips files which exist on the destination and have a modified time newer than the source file.
-p :  Preserve permissions
-o :  Preserve owner
--bwlimit=<kpbs> : To control the I/O bandwidth if you are syncing across WAN (e.g. --bwlimit= 30000  - to sync at 30 Mbps)
--delete  :   Delete all the extra files on Destination which aren’t present in the Source.
--exclude=<pattern>  :   Exclude files from syncing whose filenames matching the PATTERN  (e.g. --exclude=.zip )

Automated Rsync script:
http://smart-scripts-for-sysadmins.blogspot.com/2012/03/sample-rsync-script.html

Finding Start time of a Linux Process

In Linux, finding Starting time of a process using 'top' output will always tend to confuse us. Below is a sample 'top' output of a bunch of 'cpio' process which are running on a Server. The TIME+ column shows some time but it is quite confusing to interpret.

Following are the command-set that I use to find actual Start-time of a Process and how long it runs:

[root@linuxhost ~]# ps -eo pid,lstart,cmd | grep -v grep | grep 14115

14115 Sat Jun  2 01:15:38 2012 cpio -pdmu /NA_cdb01_ud4001

[root@iss-365-tdb02 ~]# date

Sat Jun  2 02:01:41 PDT 2012

[root@linuxhost ~]# ps -eo pid,etime,cmd | grep -v grep | grep 14115

14115       46:23 cpio -pdmu /NA_cdb01_ud4001

[root@iss-365-tdb02 ~]#

The above output shows the Process with PID 14115 started at 01:15:38 AM PDT and it is running for more than 45 minutes.

Script to generate file-system usage report

This script is developed specifically for checking the File-system usage of SAN Mounts on Linux or any UNIX flavors. 
However this can be used to check any Unix filesystem such as /var, /opt, /root etc. This script takes Access, Modified and Change Time as parameters on the specified Mount points and followed by an E-mail address. Upon execution, it will send you the resultant output of the script to the E-mail address that you specified in sorted form with the total size of matching files. In addition a copy of the script output will be stored on the path where you executed the script in the following format: Hostname-Month-Day-Year.

URL to download: http://dl.dropbox.com/u/50666315/scripts/SAN_Mount_Check.pl

Supported platforms: Any Unix platform with Perl version 5.x or above installed.

How to use:

The syntax for executing this script is as follows:

# SAN_Mount_Check.pl [-t]  [-a <value> -m <value> -c <value>]  <Mount points  separated by space>

“-t” for Total disk space

“-a” for access time

“-m” for modification time

“-c” for change time

Upon executing the above command will prompt you to enter the E-mail address to which the file-system usage report needs to be sent. Please note the switch “-t” is optional whereas the other switches “ -a, -m, -c “ are must (atleast one or more). You can give multiple filesystem pathnames separated by space but I suggest giving just one in-order to avoid confusion in sorted output; where the files will be sorted in accordance to its size.

Few Sample Syntax:

# perl SAN_Mount_Check.pl -t -a 1000 /san_mount1      <-- List Files which are accessed 1000 days before.

# perl SAN_Mount_Check.pl -t -c 1000 /Netapp_filer_mount2  <-- Files which are changed 1000 days before.

# perl SAN_Mount_Check.pl -t -m 300  /var                <-- Files modified 300 days before in /var

# perl SAN_Mount_check.pl -t -a 500 -m 750 /u01      <-- Files which are accessed 500 days before and modified 750 days before on /u01.

# perl SAN_Mount_Check.pl -t -a 365 /san_mnt1 /san_mnt2     <-- Files which are accessed 365 days before on 2 SAN mounts

How it works:

This script uses &wanted subroutine which comes along with standard Perl Module “File::Find” for traversing through all the files in a desired filesystem. For each file it finds, it checks whether it matches with the parameters supplied and if it is true, it will output the file information along with file size information.  This is a non-intrusive script, as this is equivalent to executing a ‘find’ command. 

Enabling ACL in /etc/fstab

I wish to set FACL for a file-system "/ua1003" with Read-Write permission for my user-account 'ashok'. When I try to do it, I noticed the file-system isn't mounted with ACL feature enabled. I have added the 'acl' option in /etc/fstab for that mount point and did a remount. After that, I was able to set the FACL. Below shown is the sequence which I did to make the ACL work:

[root@hostxyz ua1003]# setfacl -m u:ashok:rwx /ua1003                          
setfacl: /ua1003: Operation not supported                                                  <-- ACL not enabled
[ashok@hostxyz /]$ grep /ua1003 /etc/fstab
LABEL=/ua1003           /ua1003                   ext3    defaults        0  0        
[ashok@hostxyz /]$ vi /etc/fstab
[root@hostxyz ~]# grep /ua1003 /etc/fstab
LABEL=/ua1003           /ua1003                   ext3    defaults,acl        0  0   <-- Enabled ACL feature in /etc/fstab
[root@hostxyz ~]# mount -o remount /ua1003                                           <-- Remounting /ua1003
[root@hostxyz ~]# setfacl -m u:ashok:rwx /ua1003                                    <-- Setting FACL to the parent folder
[root@hostxyz ~]# getfacl /ua1003
getfacl: Removing leading '/' from absolute path names
# file: ua1003
# owner: applprod
# group: dba
user::rwx
user:ashok:rwx
group::r-x
mask::rwx
other::r-x
[root@hostxyz ~]# setfacl -R -m u:ashok:rwx /ua1003                               <--Setting FACL recursively
[root@hostxyz ~]# getfacl /ua1003
getfacl: Removing leading '/' from absolute path names
# file: ua1003
# owner: applprod
# group: dba
user::rwx
user:ashok:rwx
group::r-x
mask::rwx
other::r-x
[root@hostxyz ~/#

PS: To check if ACL is enabled on a given file-system, execute 'tune2fs -l <devicename>' command and look out for "Default mount options".

How to encrypt and decrypt a file in Linux ?

There could be few ways to Encrypt and Decrypt a file in Linux. The one which I use is, gpg (GnuPG).
Below shown are the steps to encrypt and decrypt a file called "confidential.txt".

[root@hostxyz ashok]# echo 'newpassXYZ' > confidentail.txt
[root@hostxyz ashok]# cat confidentail.txt
newpassXYZ
[root@hostxyz ashok]# gpg -c confidential.txt
Enter passphrase:  <secret word>
Repeat passphrase: <secret word>
[root@hostxyz ashok]# ls -l confidentail.*
-rw-r--r-- 1 root root 11 May 22 22:01 confidential.txt
-rw-r--r-- 1 root root 66 May 22 22:01 confidential.txt.gpg
[root@hostxyz ashok]# cat confidential.txt.gpg
ê2Ãà pisÃu?î^ó5Ã\<dÃ
â[root@hostxyz ashok]# mv confidential.txt.gpg /tmp
[root@hostxyz ashok]# cd /tmp
[root@hostxyz tmp]# gpg -d confidential.txt.gpg
gpg: CAST5 encrypted data
Enter passphrase:  <secret word>
[root@hostxyz tmp]# cat confidential.txt
newpassXYZ
[root@hostxyz tmp]#

The config file for 'gpg' is /root/.gnupg/gpg.conf

Finding number of occurrence of a string in a file.

Let's say you have a file (filename: testfile) as shown below and you want to find number of occurrence of a string 'snapshot' in this file.  Please note the specified string could occur more than once in a line. 

"A disk "Snapshot" is a copy of the virtual machine disk file (VMDK) at a certain point in time. It preserves the disk file system,

system memory of your VM by enabling you to revert to the snapshot in case something goes wrong. Snapshot can 

upgrading or patching applications and servers. This article will go over everything you need to know about using snapshot,

including what they are, how they work and advanced techniques. A virtual machine provides several operations for managing

snapshots and snapshot chains. These operations let you create snapshots, revert to any snapshot in the chain, and remove snapshots. You can create extensive snapshot trees."

Command is:  grep -io snapshot testfile | wc -l

Re-executing a command from History after substitution

At times we might be executing a long command on Shell prompt and later we might have to re-run that same command-set with one small change. It will be real pain to type the same command-set again just for a one parameter change. Linux has a Bash built-in command called 'fc' (stands for find command) using which we can make this task simple.

Illustration:
Let's say you have executed the following command (bit long) some time ago:
"bash /root/itc/hc/dyn/nmon -f -t -m /var/log/nmon -s300 -r iss-365-rhel5664-tmpl.xyz.com -c196"
Now you wish to execute the same command with one value changed in it. Instead of -s300, you want to re-run the command with the new value -s200.
hostxyz # fc -l               <-- Will list 16 most recent commands
427      ls
428      ls -l
429      vim NMON_startup_script.sh-May-16-12.log
430      sh /root/itc/hc/dyn/NMON_startup_script.sh
431      cat NMON_startup_script.sh-May-16-12.log
432      uptime
433      perl /opt/sysinfo.pl
434      ls -l /var/lock/subsys
435      bash /root/itc/hc/dyn/nmon -f -t -m /var/log/nmon -s300 -r iss-365-rhel5664-tmpl.xyz.com -c196
436      cd ~ashok
437      ls
438      ps -ef | grep nmon
439      kill -15 3384
440      date
441      fc -l
442      ps -ef | grep nmon
hostxyz # fc -s s300=s200 bash      
bash /root/itc/hc/dyn/nmon -f -t -m /var/log/nmon -s200 -r iss-365-rhel5664-tmpl.xyz.com -c150
hostxyz #
With the `fc -s [pat=rep ...] [command]' format, the command starting with 'bash' is re-executed after the substitution OLD=NEW is performed.
Note:  Typing just 'fc' will open the last command in Text Editor. If you want to open range of commands that you executed before in Text Editor, you can use the following syntax:  # fc [Start no]  [End no].

FUNC - Fedora Unified Network Controller

FUNC is an open source automation tool developed by using Python programming language. We can use this tool for automating system admin tasks such as status check, configuration tweak, file transfer, rebooting the systems etc in multiple Linux-based systems. It uses the typical Client-Server model, where the server is called by the term "overlord" and the clients which are binded to the server are called as "minions".

Check out this link for more details with examples:
http://advanced-sysadmin-stuff.blogspot.in/2011/03/func-fedora-unified-network-controller.html

I have developed an Installation bundle which consists of Script for installing and configuring the FUNC client with all the required packages. URL to download the software bundle : http://dl.dropbox.com/u/50666315/blog/func_pack.tar

Just download it, extract and install the FUNC client by executing the 'install_func_client.sh' script. http://dl.dropbox.com/u/50666315/scripts/install_func_client.sh

PS: If you are looking for a definitive solution on permanent basis to address all your challenges in remote administration of Linux-based servers, I would recommend to go for "Puppet" (http://puppetlabs.com) . With Puppet, you can achieve lot more things than FUNC. URL to download a book on Puppet: http://db.tt/LtZOcfZk

How to force a user to reset his password upon next login ?

Syntax     :  chage -d 0 [user-name]
Example   :  chage -d 0 ashok

How to use 'ls' command to list only directories?

# ls -ld */
(or)
# ls -lp | grep ^dr
(or)  
# ls -pl | grep /$


To display the size of each folder listed above:

# ls -ld */ | awk '{print $9}' | xargs du -sh

How to know File creation time in Linux ?

In Linux or any Unix flavors, file-creation time isn't maintained. It will get change when the file-content changes. I tested and confirmed this 1.5 years before and shared my test results in this post.
http://ashok-linux-tips.blogspot.com/2010/11/finding-file-creation-time-in-linux.html

Now the good news is, with 'EXT4' file-system it is possible to get the File-creation time.

Here's the test results performed on a EXT4 file-system:

root@Linux-Mach # df -h
Filesystem            Size    Used   Avail   Use% Mounted on
/dev/sda2             3.8G  2.2G    1.4G   62%   /
/dev/sda3             4.6G  139M  4.3G    4%     /home
/dev/sda1              46M   11M   33M   25%   /boot
tmpfs                   333M     0      333M  0%    /dev/shm
root@Linux-Mach # cd /opt
root@Linux-Mach # touch sample_file
root@Linux-Mach # ls -l sample_file
-rw-r--r-- 1 root root 0 2012-04-24 18:35 sample_file                    # File-creation time is 18:35

root@Linux-Mach # date
Tue Apr 24 18:40:12 CDT 2012
root@Linux-Mach # cat > sample_file           
This is test file to check creation time of this file.
Ctrl+D
root@Linux-Mach # ls -l sample_file  
-rw-r--r-- 1 root root 29 2012-04-24 18:40 sample_file
root@Linux-Mach # ls -i sample_file  
135528 sample_file                                                                    # inode number  
root@Linux-Mach # debugfs -R 'stat <135528>' /dev/sda2          
Inode 14552801   Type regular    Mode  0644   Flags 0x80000
Generation 340511001    Version 0x0000000000000001
User  1000   Group  1000   Size 29 
File ACL 0    Directory ACL 0 
Links 1   Blockcount 8 
Fragment  Address 0    Number 0    Size 0
 ctime 0x3f1da5b513cbff4 -- Tue Apr 24 18:40:51 2012
 atime 0x3f1da5ec8725434 -- Tue Apr 24 18:40:54 2012
 mtime 0x3f1da5b513cbff4 -- Tue Apr 24 18:40:51 2012
crtime 0x3f1cacc966104fc -- Tue Apr 24 18:35:28 2012             # we could see file creation time    
Size of extra inode fields 28
EXTENTS
(0) 44665199
debugfs 1.41.11 (14-Mar-2010)

root@Linux-Mach #

Please note the entry "crtime"  which displays the file-creation time. In my view, this information would be of great help in certain situations.

Listing and Extracting files from a RPM

As far I know, there are 3 ways available to list the files inside a RPM and they are as follows:
 To list only files:
# rpm -qlp  packagename.rpm

To list files with permission with ownership:
# rpm2cpio packagename.rpm | cpio -tv

 To list files with detailed description about the RPM:
# less packagename.rpm

Next to extract a file from a RPM, we can again use the 'rpm2cpio' command along with 'cpio' with different switches as shown as follows: 

 [root@hostxyx tmp]# rpm2cpio packagename.rpm | cpio -idvm
./etc/blkid
./etc/mke2fs.conf
./sbin/badblocks
./sbin/blkid
./sbin/debugfs
./sbin/dumpe2fs
./sbin/e2image
./sbin/fsck
./sbin/logsave
./sbin/resize2fs
./usr/bin/chattr

.

.

<Output truncated>

The above command will create directories such as ./etc, ./sbin, ./usr under the folder where RPM is kept. Please note this wouldn't install the RPM on the system, it will only extract the files. Once you grab the file that you wish to, just delete all the directories that are created. Never execute this command being in /, as it will mess up the entire file-system.

How to log commands executed by all the users in Linux?

By adding the following entry in /etc/bashrc, we can log the commands executed by all the users on a Linux machine. 
This would be certainly helpful for tracking commands on Critical servers.

PROMPT_COMMAND='history -a >(logger -t "$USER[$PWD] $SSH_CONNECTION")'

After you add the above entry at the end of /etc/bashrc file, execute the command 'source /etc/bashrc' or logout and login back to your session. Now the commands executed by all the users will be logged in /var/log/messages.

Note: If you wish to log the commands on to a different file, please check the solution given in the comments section.

Sample test result:

Apr 18 13:35:21 Linux-Mach root[/root] 192.168.4.5 51650 172.16.0.252 22: uptime
Apr 18 13:35:24 Linux-Mach root[/opt] 192.168.4.5 51650 172.16.0.252 22: cd /opt
Apr 18 13:35:26 Linux-Mach root[/opt] 192.168.4.5 51650 172.16.0.252 22: ls -lR
Apr 18 13:35:35 Linux-Mach root[/opt] 192.168.4.5 51650 172.16.0.252 22: iostat -x 2
Apr 18 13:35:39 Linux-Mach root[/root] 192.168.4.5 51650 172.16.0.252 22: cd /root
Apr 18 13:35:39 Linux-Mach root[/root] 192.168.4.5 51650 172.16.0.252 22: ls -l
Apr 18 13:35:51 Linux-Mach root[/home] 192.168.4.5 51650 172.16.0.252 22: cd /home
Apr 18 13:35:52 Linux-Mach root[/home] 192.168.4.5 51650 172.16.0.252 22: ls
Apr 18 13:35:56 Linux-Mach root[/home] 192.168.4.5 51650 172.16.0.252 22: httpd -t

Apr 18 13:51:24 Linux-Mach test1[/home/test1] 192.168.6.8 9106 172.16.0.252 22: ls -l
Apr 18 13:53:20 Linux-Mach test1[/var/lock/subsys] 192.168.6.8 9106 172.16.0.252 22: cd /var/lock/subsys
Apr 18 13:53:30 Linux-Mach test1[/var/lock/subsys] 192.168.6.8 9106 10.160.0.252 22: ls -ltr

Updated one:

# echo $PROMPT_COMMAND

RETRN_VAL=$?;logger -p local3.debug "$(whoami)  $remoteip  [$$]: $(history 1 | sed "s/^[ ]*[0-9]\+[ ]*//") [$RETRN_VAL]"

On RHEL 8.x

[root@rhel8 ]# cat /etc/profile.d/prompt.sh 

remoteip=$(who am i | awk '{print $5}' | sed "s/[()]//g")

export PROMPT_COMMAND='RETRN_VAL=$?;logger -p local3.debug "$(whoami)  $remoteip  [$$]: $(history 1 | sed "s/^[ ]*[0-9]\+[ ]*//") [$RETRN_VAL]"'

[root@rhel8 ]# 

Retrieving the file-system from a lost Linux partition

When you lost a Linux partition on Storage device, there is a 80% chance that you can retrieve the file-system in it just by Re-creating the lost Partition. I have tried couple of times in the recent past and it worked successfully.

Here's the sequence of steps which I performed to retrieve the file-system "/us2001" on a lost partition from a SAN disk:

[root@server-tap04 mapper]# mount -t ext3 /dev/mapper/us2001np1 /us2001

mount: special device /dev/mapper/us2001np1 does not exist

[root@server-tap04 mapper]# fdisk -l /dev/mapper/us2001n

 Disk /dev/mapper/us2001n: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/mapper/us2001n doesn't contain a valid partition table

[root@server-tap04 mapper]#

[root@server-tap04 mapper]# multipath -l | grep -A5 us2001n

us2001n (350002ac009290783)

[size=10 GB][features="1 queue_if_no_path"][hwhandler="0"]

\_ round-robin 0 [active]

\_ 1:0:0:1 sdc 8:32 [active]

\_ 2:0:0:1 sdb 8:16 [active]

[root@server-tap04 mapper]# fdisk -l /dev/sdc

 Disk /dev/sdc: 10.7 GB, 10737418240 bytes

 64 heads, 32 sectors/track, 10240 cylinders

 Units = cylinders of 2048 * 512 = 1048576 bytes

 Disk /dev/sdc doesn't contain a valid partition table

[root@server-tap04 mapper]# fdisk /dev/sdc

Command (m for help): n

Command action

   e   extended

   p   primary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-10240, default 1):

Using default value 1

Last cylinder or +size or +sizeM or +sizeK (1-10240, default 10240):

Using default value 10240

Command (m for help): w

The partition table has been altered!

 Calling ioctl() to re-read partition table.

Syncing disks.

[root@server-tap04 mapper]# kpartx -a /dev/mapper/us2001n

[root@ server-tap04 mapper]# kpartx -l /dev/mapper/us2001n

us2001np1 : 0 20971488 /dev/mapper/us2001n 32

[root@rica-chi-tap04 mapper]# mount -t ext3 /dev/mapper/us2001np1 /us2001

[root@rica-chi-tap04 mapper]# df -h /us2001
Filesystem            Size  Used Avail Use% Mounted on

/dev/mapper/us2001np1

                      9.9G  5.8G  4.1G  59% /us2001

[root@rica-chi-tap04 mapper]# cd /us2001                <-- Partition recreated and mounted  

[root@rica-chi-tap04 us2001]# ls -l                           <-- All the files are retrieved

total 28

-rw-r--r--  1 root     root     0 Jan 18  2011 2

-rw-r--r--  1 root     root     0 Jan 18  2011 abc

drwxr-xr-x  5 appltest dba   4096 Aug 10  2010 BI_Disco10g

drwx------  2 root     root 16384 Aug  6  2010 lost+found

drwxr-xr-x  4 appltest dba   4096 Mar 11  2011 product

[root@server-tap04 us2001]# touch xyz
[root@server-tap04 us2001]#