Help with Raid 5

[DocGreen]

Hi guys!

So I'm completely losing it over here... I have a RAID 5 consisting of 4 1TB disks setup on my fileserver using an onboard Intel RST controller. Last night while streaming some media from the server everything just stopped. I found out that one of my 4 disks died, and my raid volume was marked as failed.

First of all... WTF?! Everything I've read says that when 1 disk in a raid5 fails, the volume is marked as degraded. Mine isn't... it's marked as failed, and I'm getting ZERO option to rebuild. I've tried replacing the failed disk with a new one, but still getting nothing. The controller sees the new disk... I can even mark it as a "spare," but it won't let me use the disk to rebuild.

If anyone can help, that would be fantastic... I have irreplaceable data on there, and just found out after the fact that my backup hasn't been running properly.

 

[frostbyte5014]

Is the RAID marked as failed or is a drive marked as failed?
Also is this a hardware controlled RAID or software controlled?

Edit: Is the RAID controlled from the bios during boot up or is is controlled within the OS?

 

[300DDR]

Stop now and clone all 4 drives. Then report back with how it goes. If you can clone them all, most likely the data will all be recoverable.

 

[DocGreen]

I think that depends on your definition of hardware controlled. The motherboard has built-in RAID support via Intel RST. Here's a couple screenshots showing RAID/disk status:

1. Volume status (failed)
https://drive.google.com/file/d/0Bx1jMgVx0ElwR0xJNmdYWkdKMGs/view?usp=sharing

2. Disk status (failed)
https://drive.google.com/file/d/0Bx1jMgVx0ElwWjIxMVNMWUdlQXc/view?usp=sharing

3. 6th disk marked as "Spare"
https://drive.google.com/file/d/0Bx1jMgVx0ElwNmZmdkFXanNCdU0/view?usp=sharing

 

[DocGreen]

Stop now and clone all 4 drives. Then report back with how it goes. If you can clone them all, most likely the data will all be recoverable.

Already started with the failed drive... btw, when I put the failed raid member into my other machine to clone, the BIOS identifies it as part of a raid and marks it "incompatible." Does that mean if I put all of the disks on that machine, it would be able to access (and rebuild?) the raid, or is it locked to the controller it was created on? (fyi: other machine has an older version of the same controller)

**Correction: I will be cloning the failed drive as soon as I free up enough storage space. That raid 5 was the largest storage device I had... makes recovery a touch difficult.

 

[YeOldeStonecat]

Also is this a hardware controlled RAID or software controlled?

Software....aka "fake-RAID"

Doc...is the RAID 5 also your boot volume? Or do you have a boot volume and the RAID 5 was as second volume..just for storage?
Pretty sure you need to get into the OS and into the Intel management utility to get the rebuild going. So if your boot was also on this, snag another drive, install Winders...install the mobo drivers and the RAID utility..and then bring up the utility with that fresh 4th drive in there.

I'm not 100% sure...but I'm "pretty sure"....you won't have the rebuild options in the BIOS util for the Intel...you need to get into that full GUI utility in Windows.

 

[DocGreen]

Software....aka "fake-RAID"

Doc...is the RAID 5 also your boot volume? Or do you have a boot volume and the RAID 5 was as second volume..just for storage?
Pretty sure you need to get into the OS and into the Intel management utility to get the rebuild going. So if your boot was also on this, snag another drive, install Winders...install the mobo drivers and the RAID utility..and then bring up the utility with that fresh 4th drive in there.

I'm not 100% sure...but I'm "pretty sure"....you won't have the rebuild options in the BIOS util for the Intel...you need to get into that full GUI utility in Windows.

Correct, OS is on a separate SSD. The Winders GUI doesn't give me the option to rebuild either (see screenshots above).

 

[DocGreen]

Stop now and clone all 4 drives. Then report back with how it goes. If you can clone them all, most likely the data will all be recoverable.

Attempting to clone the failed disk w/ Clonezilla failed... "no input device" popped up in the CLI after I selected all my options and just before it started the clone. Recommend a different clone tool?

 

[Markverhyden]

Continue on with the other 3 then comeback and revisit the failed drive. Is the drive showing up in in BIOS? When you boot Clonezilla go to CLI and type dmesg to see what devices show up.

 

[HCHTech]

just found out after the fact that my backup hasn't been running properly.

Something about "shoemaker's children". I would try ddrescue, Keep track of the which drive was plugged into which connector on the original MB, and if you find that more than one drive has failed, I would send it out if your data is important. This an expensive lesson (measured in time or money, take your pick) in the importance of backup.

 

[YeOldeStonecat]

Correct, OS is on a separate SSD. The Winders GUI doesn't give me the option to rebuild either (see screenshots above).

Yeah I shouldn't clicked on the screenies....twas in a rush at home. At HQ now....but I can only see the first picture..I click on the other 2 pics and it reverts to the first pic.

Doing a quick Google...find threads that the Intel util wants you to take a fresh new drive and make it a spare. And then you have some option to rebuild once it's imported as a spare.

 

[Markverhyden]

Each picture shows a different disk/item selected. What happens when you reset the disk to available.

 

[YeOldeStonecat]

Refresh of browser got me to see diff pics now.
Normally in server RAID management utils...you have a logical view, and a physical view. Typically rebuilding is done in the logical view.

I'm just not familiar with the steps in these Intel utilities...is there a way to highlight the RAID volume, and highlight the fresh drive...and reveal some import or rebuild option?

What does that "Status" button in the upper left corner bring up?

 

[TAPtech]

I think you boot with the CTRL+I option to get into the Intel RAID config.... add the new drive as a hot spare. Then boot into windows and see if it is rebuilding in the GUI or see if the option is there.

 

[lcoughey]

Give me a call to discuss. 1-866-850-3169

Luke

Edit: this was intended to be a PM, not that it matters. I'm not trying to "get a job." I'm trying to give faster direct advice and we can post the results to the forum afterwards.

Hi guys!

So I'm completely losing it over here... I have a RAID 5 consisting of 4 1TB disks setup on my fileserver using an onboard Intel RST controller. Last night while streaming some media from the server everything just stopped. I found out that one of my 4 disks died, and my raid volume was marked as failed.

First of all... WTF?! Everything I've read says that when 1 disk in a raid5 fails, the volume is marked as degraded. Mine isn't... it's marked as failed, and I'm getting ZERO option to rebuild. I've tried replacing the failed disk with a new one, but still getting nothing. The controller sees the new disk... I can even mark it as a "spare," but it won't let me use the disk to rebuild.

If anyone can help, that would be fantastic... I have irreplaceable data on there, and just found out after the fact that my backup hasn't been running properly.

 

[jbartlett323]

Your problem is the new drive, and your screeny's tell me so. You are trying to install a drive with a physical 512 sector size, and all the currents are 4k sectors. Basically, this drive is incompatible with your array, try another.
I have worked with many Intel based arrays, and yes, if a rebuild is possible, it will give you the option in the BIOS utility. But like I said, it sees the drive as different and won't rebuild with it.
Its also not uncommon to mark an entire array "Failed" until a replacement drive is added. Intel does this for data protection, AKA so an end user wont use a degraded array.
Also, yes you should be able to hook this array to any other machine that runs the Intel RST. May need to be of similar firmware, as sometimes large discrepancies can cause issues and data loss, so not a recommended path unless your mobo is dead.

 

[Markverhyden]

Your problem is the new drive, and your screeny's tell me so. You are trying to install a drive with a physical 512 sector size, and all the currents are 4k sectors. Basically, this drive is incompatible with your array, try another.
I have worked with many Intel based arrays, and yes, if a rebuild is possible, it will give you the option in the BIOS utility. But like I said, it sees the drive as different and won't rebuild with it.
Its also not uncommon to mark an entire array "Failed" until a replacement drive is added. Intel does this for data protection, AKA so an end user wont use a degraded array.
Also, yes you should be able to hook this array to any other machine that runs the Intel RST. May need to be of similar firmware, as sometimes large discrepancies can cause issues and data loss, so not a recommended path unless your mobo is dead.

The individual drives are 512/512. It's the array itself that is 4k/512.

 

[DocGreen]

Each picture shows a different disk/item selected. What happens when you reset the disk to available.

When I reset the disk I'm given the option to mark the array as 'normal' at which point it attempts to rebuild, then fails. But... if I pull the bad disk and replace with a good disk, it doesn't give me the option to reset/mark as normal. It shows the bad drive as 'missing' and the new as a non-member.

 

[DocGreen]

Refresh of browser got me to see diff pics now.
Normally in server RAID management utils...you have a logical view, and a physical view. Typically rebuilding is done in the logical view.

I'm just not familiar with the steps in these Intel utilities...is there a way to highlight the RAID volume, and highlight the fresh drive...and reveal some import or rebuild option?

What does that "Status" button in the upper left corner bring up?

Highlighting the new disk only gives me the option to 'mark as spare' or the reverse.

 

[DocGreen]

I think you boot with the CTRL+I option to get into the Intel RAID config.... add the new drive as a hot spare. Then boot into windows and see if it is rebuilding in the GUI or see if the option is there.

I've seen screenshots where the option ROM says to boot to the OS to rebuild... but I'm not getting that.