How to run root filesystem checks on virtual servers

How to run root filesystem checks on virtual servers

Note: Server has root filesystem error on reboot
Solution: This requires root filesystem check on reboot

1. Login to vmware and open a console for the server.

2. Click on the CD icon, and connect to ISO image for current version of the OS (In our case Red Hat 64-bit)

3. Use control-D to reboot server at the console session.
Note: It may take about 5-10 minutes to complete.

4. When Linux prompt is displays, click on console and type "linux rescue"  [ At the splash menu ] at the prompt

5. Select "English" for language and "US" for keyboard

6. Select "No" to start network interfaces

7. Click "Continue" for mount filesystems (this is critical to fix the errors)

8. Click "OK" when all filesystems mounted message appears

9. Shows all mounted filesystems
Note: the name of the root filesystem device, whatever is with /mnt/sysimage only

10. Display currently mounted FS.
# mount

/proc on /proc type proc (rw)
/dev on /dev type tmpfs (rw)
/dev/pts on /dev/pts type devpts (rw)
/sys on /sys type sysfs (rw)
none on /tmp type ramfs (rw)
none on /tmp/ramfs type ramfs (rw)
/tmp/loop0 on /mnt/runtime type squashfs (ro)
/selinux on /selinux type selinuxfs (rw)
/dev/sda2 on /mnt/sysimage type ext3 (rw,data=ordered)
/dev/SYSTEM/TMP on /mnt/sysimage/tmp type ext3 (rw,data=ordered)
/dev/SYSTEM/APPS on /mnt/sysimage/apps type ext3 (rw,data=ordered)
/dev/SYSTEM/WEB on /mnt/sysimage/web type ext3 (rw,data=ordered)
/dev/SYSTEM/WEB-SERVERS on /mnt/sysimage/web/servers type ext3 (rw,data=ordered)
/dev/SYSTEM/WEB-DOCROOT on /mnt/sysimage/web/docroot type ext3 (rw,data=ordered)
/dev/sda1 on /mnt/sysimage/boot type ext3 (rw,data=ordered)
/tmp/sysfs on /mnt/sysimage/sys type sysfs (rw)
/tmp/proc on /mnt/sysimage/proc type proc (rw)
/dev/DATA_DB1/ENGS on /mnt/sysimage/engs type ext3 (rw,data=ordered)
/dev/DATA_DB2/ENGS_TS on /mnt/sysimage/engs_ts type ext3 (rw,data=ordered)
/dev on /mnt/sysimage/dev type tmpfs (rw)
/dev/pts on /mnt/sysimage/dev/pts type devpts (rw)
/selinux on /mnt/sysimage/selinux type selinuxfs (rw)

11. umount all the non-rescue, real filesystem: anything under /mnt/sysimage
# mount | grep /mnt/sysimage | awk '{print $3}' | sed -n -e '2,$p' | sort -r | awk '{print "umount " $1}'  | sh

a. There may be some stragglers
# mount
/proc on /proc type proc (rw)
/dev on /dev type tmpfs (rw)
/dev/pts on /dev/pts type devpts (rw)
/sys on /sys type sysfs (rw)
none on /tmp type ramfs (rw)
none on /tmp/ramfs type ramfs (rw)
/tmp/loop0 on /mnt/runtime type squashfs (ro)
/selinux on /selinux type selinuxfs (rw)
/dev/sda2 on /mnt/sysimage type ext3 (rw,data=ordered)
/dev/system/WEB on /mnt/sysimage/web type ext3 (rw,data=ordered)

b. Unmount those by hand
# umount /mnt/sysimage/web

12. Unmount the root filesystem last
# umount /mnt/sysimage

13. Run the check on the root filesystem
# e2fsck -y /dev/sda2

14. Disconnect the CD in vmware console

15. reboot the server
# reboot

Run vsftpd as a service

Find the process id of the application running on certain port

This script helps you to find the application running on particular port.
To execure the script, make the script executable and supply the port as a command line parameter.
# netstat -an | grep 2051 ( to see if port is used)
# sh port_proc.sh <port_num>
# sh port_proc.sh 2051 (To check the process of the PID)

root@sunserv01:/var/tmp > cat port_proc.sh
#!/bin/bash
 # is the port we are looking for
if [ $# -lt 1 ]
 then
 echo "Please provide a port number parameter for this script"
 echo "e.g. %content 2051"
 exit
 fi
echo "Greping for your port, please be patient (CTRL+C breaks) . "
for i in `ls /proc`
 do
 pfiles $i | grep AF_INET | grep $1
 if [ $? -eq 0 ]
 then
 echo Is owned by pid $i
 echo "-----------------"
 fi
 done
root@sunserv01:/var/tmp >

root@sunserv01:/var/tmp > sh port_proc.sh 2051
Greping for your port, please be patient (CTRL+C breaks) .
pfiles: cannot examine 1291: no such process
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
        sockname: AF_INET 192.168.10.184  port: 2051
Is owned by pid 6692
---------------
root@sunserv01:/var/tmp > root@sunserv01:/var/tmp > ptree 6692
6692  /opt/sybase1253/ASE-12_5/bin/backupserver -e/opt/sybase1253/ASE-12_5/install/er
root@sunserv01:/var/tmp > ptree 6692
6692  /opt/sybase1253/ASE-12_5/bin/backupserver -e/opt/sybase1253/ASE-12_5/install/er
root@sunserv01:/var/tmp >

source: http://onlineappsdba.com/index.php/2008/06/10/how-to-find-pid-listening-particular-port-on-linuxsolaris-10/

My SVN notes

A. Installation

1. Go to the site and download the file and just install the default.

http://software.opensuse.org/download.html?project=devel:tools:scm:svn&package=subversion

select your version, my version is suse 11.

Installation failed twice and finally able to install on third attempt.

2. Create a repository.

jay@suvi:~> mkdir repository
jay@suvi:~> cd repository/
jay@suvi:~/repository> ls -l
total 0
jay@suvi:~/repository>

3. Now, lets bring the directory repository under svn.

jay@suvi:~/repository> svnadmin create /home/jay/repository
jay@suvi:~/repository> pwd
/home/jay/repository
jay@suvi:~/repository> ls
conf  db  format  hooks  locks  README.txt
jay@suvi:~/repository>
jay@suvi:~/repository> more README.txt
This is a Subversion repository; use the 'svnadmin' tool to examine
it.  Do not add, delete, or modify files here unless you know how
to avoid corrupting the repository.

Visit http://subversion.apache.org/ for more information.
jay@suvi:~/repository>

Note: please note the changes on the directory structure after using svnadmin command.

As read me file suggest, do not make changes to repository unless you know what you doing.

4. Creating Project.
and start a project with a simple file.
jay@suvi:~> pwd
/home/jay
jay@suvi:~> mkdir project1
jay@suvi:~> echo "Hello This is a test Project file">project1/hello.txt
jay@suvi:~> more project1/hello.txt
Hello This is a test Project file

Note: I am also not following recommened layout of subversion project which includes branches, tags and trunk. We will be working on "trunk" of the project as the instruction doc. This is the backbone of the project where main files and directories are placed.

5. Import the project.
To import the project1 using the following command,

jay@suvi:~> pwd
/home/jay
jay@suvi:~> svn import /home/jay/project1/ file:///home/jay/repository/project1/trunk -m "Initial import of Project1"
Adding         project1/hello.txt

Committed revision 1.
jay@suvi:~>


Note: as you see, you specify by source project directory followed by destination and -m to supply the message. Always add message when you import message.

6. Now, delete the temp Project file which originally created. We already imported it to subversion.
jay@suvi:~> pwd
/home/jay
jay@suvi:~> rm -rf project1

7. Check out a repository.
Subversion keeps a copy of each lets say commit which process is lets say called check out. Check out saves the initial project with the initial version or say version 1. After that we can make changes to the project and 'check in' or commit the change. So, the any subsequent commit would be next version say 2 of the project and so on.

8. Now, create a new directory on your home directory.

Network Bonding

Assuming two network interfaces eth0 and eth1 as shown below.

[root@samserv05 network-scripts]# ifconfig -a
eth0      Link encap:Ethernet  HWaddr D8:D3:85:5F:D1:B6
          inet addr:172.28.7.206  Bcast:172.28.7.255  Mask:255.255.255.128
          inet6 addr: fe80::dad3:85ff:fe5f:d1b6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:744339 errors:0 dropped:0 overruns:0 frame:0
          TX packets:234404 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:638166148 (608.6 MiB)  TX bytes:37473617 (35.7 MiB)
          Memory:fbd60000-fbd80000
eth1      Link encap:Ethernet  HWaddr D8:D3:85:5F:D1:B7
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Memory:fbde0000-fbe00000
1. Edit modprob.conf and append “alias bond0 bonding” as shown below.
Vi  /etc/modprob.conf
alias bond0 bonding
2. Create bond0 file and append below entry in it. Change the IPADDR and NETMASK to the one on your server
cd /etc/sysconfig/network-scripts/
cp ifcfg-eth0 ifcfg-eth1 /var/tmp
vi  etc/sysconfig/network-scripts/ifcfg-bond0
ifcfg-bond0
DEVICE=bond0
IPADDR=172.28.38.134
NETMASK=255.255.254.0
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
BONDING_OPTS="mode=active-backup miimon=100"


3. Edit eth0 and eth1 and append the content below. Replace HWADDR with the hardware address on your servers
Ifcg-eth0
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
ETHTOOL_OPTS="speed 1000 duplex full"
HWADDR=00:17:A4:78:5C:B9
Ifcfg-eth1
DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
ETHTOOL_OPTS="speed 1000 duplex full"
HWADDR=D8:D3:85:5D:F1:C7
4. Run the command below from the console
First down the active eth, e.g. eth0
Ifconfig eth0 down
/etc/init.d/network restart 
[root@samserv05 network-scripts]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Speed: 100 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: d8:d3:85:5f:d1:b6
Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 1
Permanent HW addr: d8:d3:85:5d:f1:c7
If this does not work, reboot the server.

Some example of if statement - Shell scripting example

#!usr/bin/ksh
num1=20
num2=30
if [[ $num2 -gt $num1 ]]; then
    echo "The second value ${num2} is greater than the first value ${num1} ."
    exit 1
else
    echo "The sum of these number is `expr $num1 + $num2`"
    echo "Cool, enjoy !!!"
fi

-----------------------------------------------------------

if [ -d /opt/share ]
then
    /usr/bin/rmdir /opt/share 2>/dev/null
        if [ $? != "0" ]
        then
            /usr/bin/rm -r /opt/share 2>/dev/null
        fi
    /usr/bin/rm -rf /opt/share 2>/dev/null
fi
/usr/bin/mkdir -p /opt/share
/usr/bin/cp ~jay/notes/* /opt/share

-----------------------------------------------------------

if [ `whoami` != "root" ]
then
    exit 1
fi

-----------------------------------------------------------

Automate FS creation task for LVM and VxVM

bash-3.2$ cat >my_LVM
# VG    FS      Size
vg01    FS_WWW  20G
vg01    FS_WWW2 30G
vg02    FS_WWW  20G
vg02    FS_WWW2 30G
---------------------------------------------
bash-3.2$ more mylvm.sh
#!/bin/sh
# Sam
# Automate FS creation task
# Mon Feb 24 12:30:18 EST 2010
#
cat my_LVM | grep -v "^#" | while read myvg myfs mysize
do
#
echo " "
echo For LVM
        echo lvcreate -L ${mysize} -n ${myfs} ${myvg}
        echo mkfs /dev/${myvg}/${myfs}
#
echo " "
echo For Veritas Volume Manager
        echo vxassist -g ${myvg} -b make ${myfs} ${mysize} layout=nostripe
        echo " "
#
echo For vxfs
#
        echo /usr/lib/fs/vxfs/mkfs -F vxfs -o largefiles /dev/vx/rdsk/${myvg}/${myfs}
        echo " "
#
echo For UFS
#
        echo newfs -v /dev/vx/rdsk/${myvg}/${myfs}
        echo fsck -y /dev/vx/rdsk/${myvg}/${myfs}
        echo "-----------------------------------"
#       echo " "
done
#
echo
echo "Task completed !!!"
echo
bash-3.2$

---------------------------------------------
bash-3.2$ sh mylvm.sh
For LVM
lvcreate -L 20G -n FS_WWW vg01
mkfs /dev/vg01/FS_WWW
For Veritas Volume Manager
vxassist -g vg01 -b make FS_WWW 20G layout=nostripe
For vxfs
/usr/lib/fs/vxfs/mkfs -F vxfs -o largefiles /dev/vx/rdsk/vg01/FS_WWW
For UFS
newfs -v /dev/vx/rdsk/vg01/FS_WWW
fsck -y /dev/vx/rdsk/vg01/FS_WWW
-----------------------------------
For LVM
lvcreate -L 30G -n FS_WWW2 vg01
mkfs /dev/vg01/FS_WWW2
For Veritas Volume Manager
vxassist -g vg01 -b make FS_WWW2 30G layout=nostripe
For vxfs
/usr/lib/fs/vxfs/mkfs -F vxfs -o largefiles /dev/vx/rdsk/vg01/FS_WWW2
For UFS
newfs -v /dev/vx/rdsk/vg01/FS_WWW2
fsck -y /dev/vx/rdsk/vg01/FS_WWW2
-----------------------------------
For LVM
lvcreate -L 20G -n FS_WWW vg02
mkfs /dev/vg02/FS_WWW
For Veritas Volume Manager
vxassist -g vg02 -b make FS_WWW 20G layout=nostripe
For vxfs
/usr/lib/fs/vxfs/mkfs -F vxfs -o largefiles /dev/vx/rdsk/vg02/FS_WWW
For UFS
newfs -v /dev/vx/rdsk/vg02/FS_WWW
fsck -y /dev/vx/rdsk/vg02/FS_WWW
-----------------------------------
For LVM
lvcreate -L 30G -n FS_WWW2 vg02
mkfs /dev/vg02/FS_WWW2
For Veritas Volume Manager
vxassist -g vg02 -b make FS_WWW2 30G layout=nostripe
For vxfs
/usr/lib/fs/vxfs/mkfs -F vxfs -o largefiles /dev/vx/rdsk/vg02/FS_WWW2
For UFS
newfs -v /dev/vx/rdsk/vg02/FS_WWW2
fsck -y /dev/vx/rdsk/vg02/FS_WWW2
-----------------------------------
Task completed !!!
bash-3.2$

==============================================================

# cat lvm_create.sh
#!/bin/sh
# SAM Bhusal
# Automate FS creation task. This script tested and verified on Redhat Enterprise 6.x version.
# Config file entry eg.
# # file_name: LVM_WEBLOGIC
# # For Weblogic
# # vg    FS      Size(GB) Mnt point
# --------------------------------
# datavg  WEB     10     /web
# datavg  WEBDATA 15     /webdata
#
# Mon Feb 24 12:13:24 EST 2014
# Update: Thu Aug 14 09:55:12 EDT 2014
# Added the entry for mount point on config file.
# Added entry to the fstab.
#
if [ `/usr/bin/whoami` != "root" ]
then
        echo "You must be root to run this script"
        exit 1
fi
# Creating LVM.
/bin/cat LVM_WEBLOGIC | grep -v "^#" | while read myvg myfs mysize mymt
do
        echo " "
        # Check if mount point already exists.
        #/bin/df -h ${mymt}
        /bin/ls -ld ${mymt}
        if [ $? -eq 0 ]; then
                echo "Mount Point exists."
                echo "Please review the config file."
                exit 1
        fi
        /sbin/lvcreate -L ${mysize}G -n ${myfs} ${myvg}
        /sbin/mkfs.ext4 /dev/${myvg}/${myfs}
        # Create mount point
        /bin/mkdir -p ${mymt}
        # Add Entry to fstab
        echo "###########################################################################" >>/etc/fstab
        echo "/dev/${myvg}/${myfs}      ${mymt} ext4    defaults        1 2" >>/etc/fstab
        # Mount the filesystem.
        /bin/mount -a
        # Verify if the filesystems were created.
        /bin/df -h      ${mymt}
        # Check the condition if the process is successful.
        if [ $? -eq 0 ]; then
                echo "Successfully Created fileystem."
                continue
        else
                echo "Failed, please review the error"
                # exit
        fi
echo " ----------------end----------------"
done
# EOF

# cat LVM_WEBLOGIC
# file_name: LVM_WEBLOGIC
# SAM Bhusal
# For Weblogic
# vg    FS       Size(GB)  mPoint
#---------------------------------------
datavg  PKGS            20      /pkgs
datavg  WWW             10      /www
datavg  WWW_DOCROOT     10      /www/docroot
datavg  WWW_SERVERS     10      /www/servers
datavg  FMAC_TEST       10      /fmac/test
datavg  FMACDATA        10      /fmacdata