Tuesday, May 31, 2011

Splitting a file in Linux


In Linux, you might want to split a file to reduce its File size or want to extract a portion of huge file which runs into 1000+ lines.

Linux provides “split” command to do this and it is explained with an example here, which shows a log file “test.log” which consists of about 1.5 lakhs lines is splitted into 10K lines each.

[root@host01-t1 log]# cat test.log | wc -l
147815                                                                                   # The file "test.log" consists about 150K lines
[root@host01-t1 log]#
[root@host01-t1 log]# cp test.log  test.log_bkp                           # First take a backup of original file

[root@host01-t1 log]# split -l 10000 test.log splitted_file            # Splitting the test.log file into 10000 lines each with prefix “splitted_file”              
[root@host01-t1 log]#
 [root@host01-t1 log]# ls -ltr
.
.
.
-rw-r--r-- 1 root  root   26112117 Oct 26 04:20 test.log
-rw-r--r-- 1 root  root    1384098 Oct 26 04:39 splitted_fileao
-rw-r--r-- 1 root  root    1766812 Oct 26 04:39 splitted_filean
-rw-r--r-- 1 root  root    1768778 Oct 26 04:39 splitted_fileam
-rw-r--r-- 1 root  root    1769234 Oct 26 04:39 splitted_fileal
-rw-r--r-- 1 root  root    1767256 Oct 26 04:39 splitted_fileak
-rw-r--r-- 1 root  root    1764489 Oct 26 04:39 splitted_fileaj
-rw-r--r-- 1 root  root    1766383 Oct 26 04:39 splitted_fileai
-rw-r--r-- 1 root  root    1767934 Oct 26 04:39 splitted_fileah
-rw-r--r-- 1 root  root    1763855 Oct 26 04:39 splitted_fileag
-rw-r--r-- 1 root  root    1765981 Oct 26 04:39 splitted_fileaf
-rw-r--r-- 1 root  root    1767587 Oct 26 04:39 splitted_fileae
-rw-r--r-- 1 root  root    1766195 Oct 26 04:39 splitted_filead
-rw-r--r-- 1 root  root    1763062 Oct 26 04:39 splitted_fileac
-rw-r--r-- 1 root  root    1767100 Oct 26 04:39 splitted_fileab
-rw-r--r-- 1 root  root    1763353 Oct 26 04:39 splitted_fileaa
-rw-r--r-- 1 jboss jboss  27684221 Oct 26 04:39 thirdparty.log
[root@host01-t1 log]#
[root@host01-t1 log]# cat splitted_fileaf | wc -l
10000
[root@host01-t1 log]# cat splitted_fileac | wc –l           # splitted file consists of 10000 lines each.
10000
[root@host01-t1 log]#

No comments:

Post a Comment