Ramblings from MostlyChris

Tech stuff and a bit more

Browsing Posts tagged filesystem

NOTE: RAID expansions can take a few hours to complete, especially if done on-line. This assumes that new drives have been added to the RAID array and a rebuild of the array has been done. You will need to follow these steps to have the OS recognize the new space.

1. Confirm fdisk -l shows the new size of the array. It should NOT match what df -h shows since the final steps have not been completed. Ex:

# fdisk -l

Disk /dev/cciss/c0d0: 734.0 GB, 734054703104 bytes
255 heads, 63 sectors/track, 89243 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

           Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d0p1   *           1          13      104391   83  Linux
/dev/cciss/c0d0p2              14         274     2096482+  83  Linux
/dev/cciss/c0d0p3             275         405     1052257+  82  Linux swap
/dev/cciss/c0d0p4             406       53309   424951380    5  Extended
/dev/cciss/c0d0p5             406       53309   424951380+  83  Linux

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p5     267G  148G  119G  24% /
/dev/cciss/c0d0p1      99M   12M   83M  12% /boot
none                 1014M     0 1014M   0% /dev/shm
/dev/cciss/c0d0p2     2.0G   79M  1.8G   5% /tmp

2. Run 'fdisk /dev/sda' or 'fdisk /dev/cciss/c0d0'. The device that you choose depends on the configuration. The fdisk -l will have shown you which one to choose. Ex.

# fdisk /dev/cciss/c0d0

The number of cylinders for this disk is set to 89243.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help):

3. Print the partition table with 'p'. Very Important–Make note of the 'Start' cylinder of the Extended partition. You will need this in order to create the new partition. Ex.

Command (m for help): p

Disk /dev/cciss/c0d0: 734.0 GB, 734054703104 bytes
255 heads, 63 sectors/track, 89243 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

           Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d0p1   *           1          13      104391   83  Linux
/dev/cciss/c0d0p2              14         274     2096482+  83  Linux
/dev/cciss/c0d0p3             275         405     1052257+  82  Linux swap
/dev/cciss/c0d0p4             406       53309   424951380    5  Extended
/dev/cciss/c0d0p5             406       53309   424951380+  83  Linux

4. Delete the Extended partition with 'd'. Enter the partition number when asked. Ex.

Command (m for help): d
Partition number (1-5): 4

5. Print the partition table again with 'p'. Verify that the Extended partition has been deleted. Ex.

Command (m for help): p

Disk /dev/cciss/c0d0: 734.0 GB, 734054703104 bytes
255 heads, 63 sectors/track, 89243 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

           Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d0p1   *           1          13      104391   83  Linux
/dev/cciss/c0d0p2              14         274     2096482+  83  Linux
/dev/cciss/c0d0p3             275         405     1052257+  82  Linux swap

6. You will now recreate the new Extended partition with 'n'. Enter 'e' for extended partition. Enter the Start cylinder that you made note of from above. This is crucial. Make sure you enter the correct start cylinder or data loss may occur. Use the default value for the last cylinder. Ex.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
e
Selected partition 4
First cylinder (406-89243, default 406): 406
Last cylinder or +size or +sizeM or +sizeK (406-89243, default 89243):
Using default value 89243

7. Create a primary partition on the new Extended partition with 'n'. Use the start cylinder noted above. Again, get this number correct or data loss can occurr. Ex.

Command (m for help): n
First cylinder (406-89243, default 406): 406
Last cylinder or +size or +sizeM or +sizeK (406-89243, default 89243):
Using default value 89243

8. Print the partition table with 'p' and verify the new Extended and Primary partitions are there and they show the correct size. Ex.

Command (m for help): p

Disk /dev/cciss/c0d0: 734.0 GB, 734054703104 bytes
255 heads, 63 sectors/track, 89243 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

           Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d0p1   *           1          13      104391   83  Linux
/dev/cciss/c0d0p2              14         274     2096482+  83  Linux
/dev/cciss/c0d0p3             275         405     1052257+  82  Linux swap
/dev/cciss/c0d0p4             406       89243   713591235    5  Extended
/dev/cciss/c0d0p5             406       89243   713591203+  83  Linux

9. If everything looks good, write the changes and exit fdisk with 'w'. Ex.

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

10. Reboot the server.

shutdown -r now

11. Once the server comes back up you will need to perform this final command 'ext2online /'. This will take a few minutes depending on the size of the expansion. Ex.

ext2online /  –or–
resize2fs /dev/cciss/c0d0p5

12. When the above command completes, run df -h and make sure the partitions are all showing the correct size of the new RAID array. If so, you are done. Ex.

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/cciss/c0d0p5     670G  148G  488G  24% /
/dev/cciss/c0d0p1      99M   12M   83M  12% /boot
none                 1014M     0 1014M   0% /dev/shm
/dev/cciss/c0d0p2     2.0G   78M  1.8G   5% /tmp

I routinely deal with Read Only file systems and always have to remember the commands to remove the journal and run a file system check. Since some days I am better at remembering than others, I thought I'd just write it down in this post so I can find it later.

First, remove the journal:

sh#   tune2fs -O ^has_journal /dev/hda5

If you run into errors due to 'needs_recovery', do this to remove the recovery flag and then remove the journal as shown above:

sh# debugfs -w /dev/hda5
debugfs: features ^needs_recovery
debugfs: quit

Run the file system check. See how bad things are first.

sh#  fsck -n /dev/hda5

If it's not too bad, do the real fsck.

sh# fsck -y /dev/hda5

Once the rebuild is successful and complete, rebuild the journal.

sh#  tune2fs -j /dev/hda5

Mount the filesystem and make sure it's happy and ready for action.

sh#  mkdir /mnt/fixed && mount -t ext3 /dev/hda5 /mnt/fixed

Reboot the server and put it back in the fray.