RAID in practice: expanding an array from 8 to 24 TB and replacing a failed drive with zero data loss

In short

We describe two separate cases from our work with RAID arrays:

1. RAID array expansion for a large accounting and audit firm: usable capacity went from 8 to 24 TB with a RAID1 to RAID5 migration on the same server. It took 1-2 working days, the server was online 99% of the time, and no data was lost. The client also had a separate backup of its data, but we never had to use it.
2. Failed RAID drive replacement for another client: our monitoring caught the problem early and the drive was replaced in a planned way - the server was online 99% of the time, with zero data loss.

Case 1: expanding a RAID array from 8 to 24 TB

The situation

The firm keeps its team's documents and data on a Linux server with software RAID1 (mdadm). RAID1 keeps a full copy of the data on two drives - very reliable, but the usable capacity equals a single drive. As the firm grew, 8 TB was no longer enough - a common story, because files grow faster than it seems when you choose a server.

The usual options were a new server, or rebuilding the storage and bringing the data back from backup. Both are costly, slow and stop the team's work.

What the client wanted

• more usable capacity on the same server;
• protection against a drive failure during and after the migration;
• no rebuild of the array and no restore from backup;
• as little disruption to daily work as possible.

How we did it

1. Checking the array. We reviewed the mdadm configuration and the health of the drives. We only start a migration on a fully healthy, synchronised array. The client also had a separate, regularly updated backup of its data - an important safety net for any change to the array.
2. Preparing the new drives. We confirmed they had enough capacity for the existing data and the new configuration.
3. In-place RAID1 to RAID5 migration. Using mdadm, we converted the existing array without copying the data to a new system.
4. Fitting and adding the drives. We fitted the new drives - the server was stopped for only a few minutes for this. We then added them to the array and monitored the reshape and resynchronisation throughout, while the server kept running and the team kept access to its files.
5. Growing the filesystem. After the resynchronisation, we expanded the array, the partitions and the filesystem, and updated the server's boot settings.

The result

• Usable capacity from 8 to 24 TB - three times more, without a new server.
• The migration was completed in 1-2 working days.
• The server was online 99% of the time - it was stopped for only a few minutes to fit the new drives.
• 0 data lost; settings and applications preserved.
• Protection against a single drive failure throughout.
• The separate backup remained just a safety net - we never had to use it.

Case 2: planned replacement of a failed RAID drive

The situation

For another client, the Linux server with software RAID (mdadm) had two physical drives and several RAID1 arrays built on partitions of those two drives. With this layout, the failure of one drive affects all arrays at once.

How monitoring caught the problem

Our 24/7 monitoring flagged a problem with one of the drives while the arrays were still running on the other one and before the client noticed anything. One of our administrators reviewed the server, confirmed the failure from the array status ([U_] instead of [UU]) and the system logs, and identified the serial number of the faulty drive. That early alert is what made a planned replacement possible.

How we did it

1. We marked the failed drive's partitions as failed and removed them from every affected array.
2. We brought a suitable replacement drive on site, at least the same size (even a slightly smaller drive prevents the partitions from being recreated), and swapped it. The server was stopped for only a few minutes, just long enough to fit the new drive.
3. We recreated the partition layout from the healthy drive and added the partitions back to their arrays.
4. We monitored the rebuild until every array was fully synchronised.

The result

• The server was online 99% of the time - it was stopped for only a few minutes to fit the new drive.
• Zero data loss, with no restore from backup.
• The problem was caught by monitoring before the client noticed anything.
• All affected RAID1 arrays fully synchronised.

24/7 monitoring is included in all our support plans.

What the two cases show

• More space does not always mean a new server. An existing software RAID can be expanded in place if it is healthy and the operation is planned.
• A failed RAID drive does not have to mean long downtime. When monitoring catches the problem early, the drive is replaced in a planned way with a stop of just a few minutes. And on an enterprise-grade server with hardware RAID and hot-swap drives, the drive can be replaced with no stop at all.
• Good preparation is half the job. Before any change, we make sure the array is healthy, the drives are suitable and there is a current backup. That way a routine operation does not turn into a problem. For more serious cases, every company also needs a disaster recovery plan.

Frequently asked questions

Can RAID1 be converted to RAID5 without data loss?

Yes. With mdadm, software RAID1 can be converted to RAID5 in place while the data stays on the array. The conditions are a healthy, synchronised array, suitable new drives and a current backup of the data.

How long does it take to expand a RAID array?

It depends on the capacity and the drives. In Case 1, expanding from 8 to 24 TB took 1-2 working days, and the server was online 99% of the time - it was stopped for only a few minutes to fit the new drives.

How can you tell that a drive in a RAID array is failing?

Monitoring tracks the state of the arrays and drives and raises an alert immediately. In Linux, a healthy two-drive array shows as [UU], and an array with a missing drive as [U_].

Does the new drive have to be the same model?

Not necessarily, but it must be at least the same size. Even a slightly smaller drive prevents the partitions from being recreated.

Can a drive be replaced with no server downtime at all?

Yes. On an enterprise-grade server with a hardware RAID controller and hot-swap drives, the failed drive is pulled and replaced while the server keeps running - zero downtime. In the two cases described here, the servers had neither hardware RAID nor hot-swap drives, which is why their protection is built with software RAID using mdadm - the standard Linux tool that gives reliable data protection without an expensive RAID controller. The only trade-off was a stop of a few minutes to fit the drives. When we plan a new server or an upgrade, we advise clients when this kind of hardware is worth it.

Conclusion

Both cases ended with zero data loss: in the first, the array was expanded from 8 to 24 TB on the same server; in the second, the failed drive was replaced with only a few minutes of planned downtime because monitoring caught it early. Operations like these are part of our server support. See also how we saved the data from a failing NVMe SSD for another client in the same sector.

If your company's server is close to its limit or running with reduced protection, let's talk about your needs and we will offer you the best solution and price.