ESM Multipath SAN v2.1
- Pavan Raja
- Apr 8, 2025
- 8 min read
Summary:
The provided text is a detailed guide on how to set up and migrate data for Oracle using logical volumes on Linux. It covers both the initial setup of a Logical Volume (LV) with LVM (Logical Volume Management) and the subsequent steps required when migrating from an old storage area network (SAN) location to a new one. Here’s a structured breakdown of the process:
### Step 1: Setting Up Logical Volume for Oracle
#### Creating the Volume Group ```bash vgcreate ARDATA /dev/mpath/mpath0p1 /dev/mpath/mpath1p1 ``` - This command creates a volume group named `ARDATA` using the specified physical volumes (`/dev/mpath/mpath0p1` and `/dev/mpath/mpath1p1`).
#### Creating Logical Volume ```bash lvcreate -L 2G -n mylv ARDATA ``` - This command creates a logical volume named `mylv` with a size of 2GB within the `ARDATA` volume group.
### Step 2: Formatting and Mounting the Logical Volume
#### Formatting with ext4 ```bash mkfs.ext4 /dev/ARCDATA/lvol0 ``` - This command formats the logical volume `/dev/ARCDATA/lvol0` with the ext4 filesystem.
#### Tuning the File System (Optional) ```bash tune2fs -c -1 -i 0 /dev/ARCDATA/lvol0 ``` - This optional command reduces mandatory fsck checks, suitable for production environments to improve performance.
### Step 3: Setting Up Mount Point and Permissions
#### Create Mount Point ```bash mkdir -p /opt/SAN ```
#### Change Ownership ```bash chown oracle:oracle /opt/SAN ```
### Step 4: Labeling the Device (Optional) ```bash e2label /dev/ARCDATA/lvol0 /opt/SAN ``` - This command labels the device with `/opt/SAN`. While optional, it helps in standardization and compliance.
### Step 5: Adding to fstab for Automatic Mounting ```bash echo "LABEL=/opt/SAN /opt/SAN ext4 defaults 1 3" >> /etc/fstab ``` - This command adds an entry to `/etc/fstab` for automatic mounting at boot time.
### Step 6: Mounting the Volume ```bash mount /opt/SAN ```
### Migrating Data from Old SAN Location
#### Mount the New SAN Volume ```bash mount /opt/SAN ``` - Ensure permissions are checked and adjusted if necessary by reissuing `chown oracle:oracle /opt/SAN`.
#### Migrate Current Data Files 1. Stop Manager service. 2. Shutdown database with `sqlplus / as sysdba shutdown immediate`. 3. Copy data files from `/opt/data/arcsight` to the new SAN location `/opt/SAN`: ```bash cp -p /opt/data/arcsight/* /opt/SAN/ ```
#### Update Oracle Data File Locations 1. Start database with `STARTUP MOUNT`. 2. Run SQL commands to update file locations: ```sql ALTER DATABASE RENAME FILE '/opt/data/arcsight/ARC_EVENT_DATA_000.dbf' TO '/opt/SAN/ARC_EVENT_DATA_000.dbf'; ALTER DATABASE RENAME FILE '/opt/data/arcsight/ARC_EVENT_DATA_001.dbf' TO '/opt/SAN/ARC_EVENT_DATA_001.dbf'; ALTER DATABASE RENAME FILE '/opt/data/arcsight/ARC_EVENT_DATA_002.dbf' TO '/opt/SAN/ARC_EVENT_DATA_002.dbf'; ``` 3. Validate the migration with `SELECT name FROM V$DATAFILE;`.
#### Start the Database and Services 1. Run `ALTER DATABASE OPEN` to start the database. 2. Restart Manager and Web services: - Manager: `service manager start` - Web: `service webserver start`
### Extend Tablespaces (Optional) ```bash ./arcsight database xts ``` - Follow the Database extend tablespace wizard to create additional files. Ensure that both the Manager and Web services are shut down during this process.
This structured guide provides a comprehensive roadmap for setting up Oracle with logical volumes on Linux, including data migration from an old SAN location. Each step is detailed with commands and considerations for practical implementation.
Details:
This guide outlines a comprehensive procedure to migrate an existing ESM/Oracle setup from local storage to a multipath SAN. The authors, Eric Siskonen and Duy Tran, provide detailed steps focusing on the migration process rather than the upgrade itself. Key components include upgrading the operating system (from Oracle Enterprise Linux v4.0 to v5.5) and the ESM software version (from 5.0.0.6450.0 to 5.2.0.6847.0), as well as preparing the SAN with multipathing for improved storage performance. The guide also covers essential tasks such as backing up system tables, setting up partitions and logical volumes, configuring SAN configurations, and formatting these for use in the new storage environment.
The task involves migrating local database data to SAN storage, updating Oracle with new database file locations, extending tablespaces, backing up system tables, and running DBCheck. Additionally, it includes upgrading OEL (Oracle Enterprise Linux) on hardware appliances, upgrading the ArcSight Management Suite (ESM), and preparing for SAN storage considerations.
1. **Backup System Tables**: Always start by backing up the customer’s system tables on the database server. This is crucial before any upgrades to preserve a copy of the backup file named `arcsight.dmp`. After this, run `dbcheck` (specifically `./arcsight dbcheck`) on the DB server to prepare for upgrades.
2. **Migration and Extending Tablespaces**: Migrate local database data to SAN storage and extend the tablespaces if necessary to accommodate the new data or updates.
3. **Upgrade OEL on Hardware Appliances**: This involves obtaining a script from ArcSight’s update depot, which is used to upgrade E7200 appliances from Oracle Enterprise Linux (OEL) version 4.0 to version 5.5. This sets the foundation OS necessary for Oracle 11G and ESM v5.2.
4. **Upgrade ESM**: The script provided will guide the upgrade of ESM from versions 5.0 GA to 5.2 GA, including upgrading the database, manager, web, and console components in that order. Ensure you have the correct binaries ready for this process, particularly Oracle 11.2.0.2 binaries for the database upgrade and 5.2.0.6847.0 binaries for ESM v5.2 GA upgrades.
5. **SAN Storage Preparation**: When setting up SAN storage infrastructure, consider factors such as HBA card speeds, fabric switch connections, fiber connection speed to the SAN, device zoning on the fabric switches, and average IOPS (Input/Output Operations Per Second). These considerations are crucial for ensuring optimal performance in data handling.
In summary, this process involves several key steps: preparation, migration, upgrade procedures, and infrastructure setup based on specific requirements of each component involved in Oracle database management and storage solutions.
The text provided discusses various aspects of configuring a storage environment, particularly for systems using RAID, SANs, and specific Linux distributions like RedHat 5.6. Here's a summary of key points:
1. **RAID Level and Volume Creation:**
It is recommended to create volumes at RAID 10 level on the SAN.
The size and number of volumes depend on the capacity and ease of creation, with larger volumes being preferable for easier logical volume management.
Settings in volume creation can vary depending on the SAN's allocation method (e.g., 3Par SAN allowing space from entire SAN, enclosure, or magazine).
2. **Server Infrastructure:**
Internal RAID: Typically sized at around 1.5TB, intended for Oracle redo logs to reduce overhead on the SAN.
HBA Cards: Use the fastest speed supported by the SAN; faster cards can accommodate potential future upgrades without issues.
3. **Linux Installation and Configuration:**
**Multipath Tools:** For RedHat 5.6, install `device-mapper-multipath-0.4.7-47.el5.x86_64.rpm` from the installation media. This setup is necessary for creating device mapper devices used in further configurations.
**LVM (Logical Volume Management):** Ensure LVM2 (`lvm2-2.02.74-5.el5.x86_64.rpm`) is installed, which is required for managing disk space and volumes effectively.
**e4fsprogs:** If ext4 is planned as the partition type for SAN volumes, install `e4fsprogs-1.41.12-2.el5.x86_64.rpm` from the installation media to support ext4 filesystem operations.
4. **Configuring and Formatting:**
After creating SAN volumes and presenting them to the server's HBAs, configure multipathing on the server using `multipath-tools`. This setup involves setting up device mapper devices that are essential for subsequent configurations.
These points highlight the importance of proper configuration and compatibility with specific Linux distributions when dealing with RAID, SANs, and storage management tools like LVM and multipath tools.
The provided summary outlines how to configure and verify multipath tools on an ArcSight appliance, focusing on adjusting the `multipath.conf` file for optimal operation without explicitly blacklisting internal RAID paths. After configuring multipathing with basic settings, start it using `/etc/init.d/multipathd start` and ensure it starts at boot time by running `chkconfig multipathd on`. To check if multipathing is functioning correctly, execute the command `multipath -l`, which should return information about active multipath devices.
Lastly, instructs creating a partition table for each SAN volume; use `fdisk` for standard DOS partition tables unless the volume exceeds 2TB, in which case create a GUID partition table.
To set up a logical volume on Linux, follow these steps after installing the `parted` package and creating partitions using GNU Parted:
1. **Install parted**: Ensure that you have installed the `parted` package from your Linux distribution's repositories or installation media. This tool is necessary for managing disk partitions.
2. **Create a GUID Partition Table**: Use the following commands to create a new GPT partition table on your volume using `parted`:
```bash
parted /dev/dm-0
mklabel gpt
```
3. **Install kpartx**: Install the `kpartx` utility for mapping partitions, which is useful when dealing with logical volumes.
4. **Create Partitions**: Use `parted` to create a single Linux partition on your volume:
```bash
parted /dev/dm-0
mkpart primary 1 -1
```
5. **Map Partitions**: Create the partition mapping for each volume using `kpartx`:
```bash
kpartx -a /dev/mpath/mpath0
kpartx -a /dev/mpath/mpath1
```
6. **Create Logical Volume**: Use LVM commands to create the logical volume:
**Physical Volume Creation**:
```bash
pvcreate /dev/mpath/mpath0p1 /dev/mpath/mpath1p1
```
**Volume Group Creation**:
```bash
vgcreate ARCDATA /dev/mpath/mpath0p1 /dev/mpath/mpath1p1
```
**Logical Volume Creation**:
```bash
lvcreate -L 2G -n mylv ARDATA
```
These steps guide you through setting up a logical volume on Linux, ensuring that each command is executed in the correct order and with the appropriate parameters to create and manage partitions and logical volumes effectively.
The provided text is about setting up a logical volume for use with Oracle, recommending ext4 filesystem due to its widespread acceptance and future improvements. It outlines steps to format the logical volume with ext4, tune the filesystem to avoid mandatory fsck checks, create a mount point, adjust permissions, label the device, and add an entry to /etc/fstab for automatic mounting at boot time.
The summary of this process is as follows:
1. **Formatting the Logical Volume**: Use `mkfs.ext4` on `/dev/ARCDATA/lvol0` to format it with ext4 filesystem. Optionally, tune the filesystem by setting `-c -1 -i 0` in `tune2fs` to reduce mandatory fsck checks unless necessary for performance reasons.
2. **Setting Up Mount Point**: Create a mount point at `/opt/SAN` using `mkdir -p /opt/SAN`. Ensure Oracle has read/write permissions by running `chown oracle:oracle /opt/SAN`.
3. **Labeling the Device**: Assign a label to the volume with `e2label /dev/ARCDATA/lvol0 /opt/SAN` for standardization and compliance, though it is optional.
4. **Adding to fstab**: Add an entry in `/etc/fstab` like `LABEL=/opt/SAN /opt/SAN ext4 defaults 1 3` for automatic mounting at boot.
5. **Mounting the Volume**: Execute commands from previous steps to prepare and mount the logical volume, making it ready for use with Oracle setups.
To migrate data from an old storage area network (SAN) location to a new one, follow these steps:
1. **Mount the New SAN Volume**: Use the command `mount /opt/SAN` to mount the volume. Ensure you pay attention to permissions on the mount point, as they may change after mounting a new volume. If necessary, reissue the `chown` command to maintain persistent ownership between mounts and unmounts.
2. **Migrate Current Data Files**:
Stop the Manager service.
Shutdown the database using `sqlplus / as sysdba shutdown immediate`.
Switch user to Oracle with `su – oracle`, then copy current data files from `/opt/data/arcsight` to the new SAN location `/opt/SAN`. Use the command `cp –p /opt/data/arcsight/* /opt/SAN/`.
3. **Update Oracle on New Data File Locations**:
Start the database with the SQL*Plus command `STARTUP MOUNT`.
Run the following SQL commands to update Oracle’s location of the new data files:
```sql
ALTER DATABASE RENAME FILE '/opt/data/arcsight/ARC_EVENT_DATA_000.dbf' TO '/opt/SAN/ARC_EVENT_DATA_000.dbf';
ALTER DATABASE RENAME FILE '/opt/data/arcsight/ARC_EVENT_DATA_001.dbf' TO '/opt/SAN/ARC_EVENT_DATA_001.dbf';
ALTER DATABASE RENAME FILE '/opt/data/arcsight/ARC_EVENT_DATA_002.dbf' TO '/opt/SAN/ARC_EVENT_DATA_002.dbf';
``` Repeat this step for each data file that has been moved.
Validate the migration by running `SELECT name FROM V$DATAFILE;` to check if it returns the list of newly migrated datafile directories.
4. **Start the Database**: Once validated, run `ALTER DATABASE OPEN` to start the database. Then, restart the Manager and Web services to ensure successful migration.
5. **Extend Tablespaces**: With the new SAN storage available, extend your tablespaces by running `./arcsight database xts` in the database server’s bin directory. Follow the Database extend tablespace wizard to create additional files. Ensure that both the Manager and Web are shutdown during this process.
Comments