Wednesday, February 1, 2012

Steps to handle a Read-Only file-system problem

Try these commands at your own risk !

Lets assume the device name of the mounted file-system on "/data1" which got into Read-only mode is "/dev/sda5".
#  mount -o remount,rw /data1   (provided fstab have the entry for mount point)
If this works, well and good. Otherwise you need to go through the pain of trying the following commands:
#  hdparm -r 0 /dev/sda5           <-- To turn-off the Read-only mode.
 # blockdev --setrw /dev/sda5   <-- To set the block (mounted) device to Read-Write mode.
If the above commands didn't the fix the problem, then go with the regular fix:
1. Check the processes that are accessing the mount point.
2. Request the process/application owner to stop the processes that are accessing the mount point.
3. Try to un-mount the filesystem (without any option).
4. Mount the file-system back on its mount point.
5. If Step 4 didn't work,  run a fsck on the un-mounted file-system (without any option).Otherwise continue Step 6.
6. Verify the mounted file-system is in Read-Write mode.
Incase if the step 3 doesn’t work:
i)  Check the processes again that are accessing the mount point and ask the Application owner to confirm that the stopped the services properly and ask them to kill any hung sessions.
ii)    Try to unmount again without any option.
iii)   If it still didn’t work, use lazy option “umount -l”   (if this works, continue from Step 4).
iv)   If lazy option didn’t work, we are in a chaotic  situation.  Try Forced un-mount by using “umount -f”  & continue from Step 4.
v)    After the above step, incase you couldn’t mount the file-system back, the best choice would be to go for a System reboot. (I’m not sure in this situation, forced Mount using ‘mount -f’ will work). 

PS: Please note this procedure will not be applicable for OS file-systems such as /, /var, /opt etc. In that case, a system reboot would be the only solution, if the remount doesn't work. 

1 comment:

  1. Very good article. I absolutely love this site. Stick with
    it!

    My blog post: same day cash advance online

    ReplyDelete