Fab's AutoBackup 7 Pro - a must have tool for techs

[fabs]

Well, at this time of the process, the only thing I may be able to add is some timeout while listing since there's already one while copying

 

[britechguy]

Aren't exception cases a grand PITA, no matter where or when?!

When all goes according to plan and smoothly far less of the code gets hit than when all sorts of exception handling come into the picture!

 

[Metanis]

I appreciate the response. That being said, based on what @Larry Sabo described happening, it seems to me like the number of retries needs to be adjusted down, way down. Or, if retries is not the way to look at it, the amount of time taken on retries needs to be reduced significantly.

I think if @fabs is getting into the data recovery business instead of just the data copying business he's going to need to increase his software rates a LOT. I understand the frustration when things go badly but when evidence of disk damage appears it's time to make a clone, then fix the clone, then pull the data after the sectors are fixed.

 

[fabs]

Aren't exception cases a grand PITA, no matter where or when?!

When all goes according to plan and smoothly far less of the code gets hit than when all sorts of exception handling come into the picture!

Could not more agree

 

[britechguy]

I understand the frustration when things go badly but when evidence of disk damage appears it's time to make a clone, then fix the clone, then pull the data after the sectors are fixed.

Well, I think there can be a happy medium.

Sometimes you don't necessarily know that there is any disk damage, as the evidence presents itself during mass copy, and if that's already in progress, having some reasonable (and I know that definition is different to different people) timeout mechanism in place when it's encountered "on the fly" whenever it might be encountered is that happy medium.

 

[nlinecomputers]

I think if @fabs is getting into the data recovery business instead of just the data copying business he's going to need to increase his software rates a LOT. I understand the frustration when things go badly but when evidence of disk damage appears it's time to make a clone, then fix the clone, then pull the data after the sectors are fixed.

This. I think everyone is asking to fix something that isn’t really broken. It’s beyond the normal operating conditions for the utility.

 

[britechguy]

I think everyone is asking to fix something that isn’t really broken. It’s beyond the normal operating conditions for the utility.

Something being broken, yet being beyond the typical operating conditions for a given thing, are not mutually exclusive.

If Fabs is hanging for an inordinate amount of time when it encounters "something off" during it's normal processing that, to me, is broken. Given the volumes of data anticipated, and the distinct possibility of a "less than perfect" source drive in many circumstances, handing an unreadable condition with grace should be expected.

This isn't a direct criticism of Fabs, specifically, but I find most copying utilities (regardless of specific type) devote way, way, way too much time to retrying when a fault is encountered rather than, after a very finite (100 or less) number of tries, moving along. We all know that if you do not get something to read from disc in the first couple of tries, the probability that it will ever read is very small indeed. That's why I like things to have very low retry counts/times whenever possible. When conditions are normal, it just doesn't matter and never comes into the picture. When an exception condition on reading is hit, it allows the process to continue in a timely manner and a error message can be presented, whether during processing, at the end of processing, in the log, or all of the above.

 

[nlinecomputers]

Well, I think there can be a happy medium.

Sometimes you don't necessarily know that there is any disk damage, as the evidence presents itself during mass copy, and if that's already in progress, having some reasonable (and I know that definition is different to different people) timeout mechanism in place when it's encountered "on the fly" whenever it might be encountered is that happy medium.

If you must have this a skip button probably is the best answer.

 

[fincoder]

On a similar topic I find it frustrating that Fabs will continue trying to copy every file even if the drive drops out completely. Does a file copy failure not give any indication of the drive not being present? Maybe there's an easy way to check after a file copy failure if the volume is even present, then Fabs could prompt asking to retry or abort.

I've had this happen because of sick drives, or dodgy USB ports.

I don't see these potential features being discussed today as data recovery features, but as improvements to error handling. A large percentage of source drives being used with Fabs will have bad sectors or other problems, that are perhaps unknown to the user (e.g. not showing in a SMART check).

 

[fabs]

On a similar topic I find it frustrating that Fabs will continue trying to copy every file even if the drive drops out completely. Does a file copy failure not give any indication of the drive not being present? Maybe there's an easy way to check after a file copy failure if the volume is even present, then Fabs could prompt asking to retry or abort.

I've had this happen because of sick drives, or dodgy USB ports.

This! That's a real concern here and there's clearly an improvement to make. Handling troubles while (just) listing files to copy is probably not the right thing to chase as this sounds like an alarm saying that data may be at risk. Since Fab's can have to adjust permissions on folders (so it writes on the disk somehow) if it can't get what's inside, then it's better to work with a clone instead of letting it cause even more damage.
The fact that there is already a retry feature while copying with a timeout before skipping to next files sounds like a good compromise to me.

Having it trying to to copy files while source drive has dropped is a real problem and definitely needs to be addressed.

 

[fabs]

That's OK. I've added some piece of code that will check if source drive (or network path) still exists if a source file is not found when trying to copy it. Then it will show a message saying that source media is missing and may be disconnected. It will give the choice to continue or cancel current job.

Edit : I've went a bit further as there was some things I did not think about. If running in command line mode, the program cannot stay just like this, waiting for someone to click the "Continue" or "Cancel" button. So, I've left it for interactive mode. In command line, it will just continue to try to do what it's doing and log the errors. Also, doing such changes, I noticed that percentage calculation of failed files was not based on the right numbers so I've fixed that too.

All of these will be in next update (hopefully very soon).

 

[fincoder]

This! That's a real concern here and there's clearly an improvement to make. Handling troubles while (just) listing files to copy is probably not the right thing to chase as this sounds like an alarm saying that data may be at risk. Since Fab's can have to adjust permissions on folders (so it writes on the disk somehow) if it can't get what's inside, then it's better to work with a clone instead of letting it cause even more damage.
The fact that there is already a retry feature while copying with a timeout before skipping to next files sounds like a good compromise to me.

Having it trying to to copy files while source drive has dropped is a real problem and definitely needs to be addressed.

When I got this in a notification email I didn't realise it was Fab himself. I thought it was another user piling on Fabs!

I've added some piece of code that will check if source drive (or network path) still exists if a source file is not found when trying to copy it.

That's fantastic.

It will give the choice to continue or cancel current job.

Will it retry the failed file if the user says to continue? That's what I'd like, or even better: change the options to Retry, Skip, Cancel (or Abort).

 

[fabs]

Will it retry the failed file if the user says to continue? That's what I'd like, or even better: change the options to Retry, Skip, Cancel (or Abort).

Then I have to go back to code as there's just a "continue" button that acts like some kind of "skip" and a "cancel" one. A "retry" is a bit trickier to add but it should be there anyway.

 

[fabs]

Well, here's what I could have now :

This is what happens when I plug out USB source drive while coping data. First, I clicked the retry button but since drive was still dropped, it failed to process it and then skipped to next file. This time, I clicked the "skip" button and it tried next file. Since I did not plus the source drive back, it stopped again asking me what to do next.
Then, even if this is not a really good idea in my opinion, if I can get source drive back, I can resume the copy job.

Sounds pretty fair to me.

 

[fincoder]

The dialog box looks good, just as I imagined.

This is what happens when I plug out USB source drive while coping data. First, I clicked the retry button but since drive was still dropped, it failed to process it and then skipped to next file.

I don't understand why a retry skipped to the next file. The user should be able to try Retry a few times without skipping any files

Since I did not plus the source drive back, it stopped again asking me what to do next.

I don't understand why it would do this. Are you checking for the drive's existence after every failed copy?

Then, even if this is not a really good idea in my opinion, if I can get source drive back, I can resume the copy job.

I might be misunderstanding, but couldn't that potentially skip files? Isn't there a way to retry a few times without skipping any files?

 

[fabs]

The dialog box looks good, just as I imagined.

I don't understand why a retry skipped to the next file. The user should be able to try Retry a few times without skipping any files

I don't understand why it would do this. Are you checking for the drive's existence after every failed copy?

I might be misunderstanding, but couldn't that potentially skip files? Isn't there a way to retry a few times without skipping any files?

It skipped after second try failed. I thought that if it still does not work, it has to move on. Obviously that's not what is expected there...
Yes, if it can't find source file, then it checks if the drive is there because it could have dropped again. I think that's relevant in such a case so I'll keep that.

Anyway, I need to rethink this as this is clearly not as fair as I thought

 

[fabs]

So, I'm polishing it.
If the source drive drops, the program will write that in the log, then it will prompt with 4 choices : "Retry", "Skip", "Skip all" (so next files in the same case will be skipped too) and "Cancel".
If we hit the "Retry" button, it will prompt again in case of another failure, and that without limited number of attempts. So, that's possible to click "Retry" again and again and again...

I've still a glitch to sort out : when retrying to copy such a failed file, there is some piece of empty destination file at 0kb that the program does not free. Because of that, even if source disk goes back on, it never gets overwritten with the correct one. Work in progress.

Edit: I've found what caused the glitch. It was because of first source disk drop : the program was supposed to grab information from source file at this moment but since it was not there, this task was in error state and the rest of the process could not be done. I've added some instructions so it will move on and free resources. Now it works fine.

 

[fincoder]

Great work @fabs

 

[fabs]

Hello,
Fab's AutoBackup 7 Pro and Fab's AutoBackup 7 Home & Office 7.12.2.6047 have just been released.

Here is the change log for both versions:

Added :
- PRO ONLY: If source drive got disconnected while processing files copy, the program continued to try to copy every still pending file. Now, in interactive mode, it will ask for retry to copy the file, skip this file to next one, skip all files in the same case or cancel current job. In command line mode, it will just try to continue and will log the errors.

Fixed:
- Failed files percentage calculation was not based on the right numbers.
- While listing files to be copied the program could look for items that are in excluded folders. It would not list for copy them but that could be annoying and quite time consuming, especially when dealing with failing hard drives that would slow down the entire process.

Grab it from your orders history's details on the shop's website at https://archive.fpnet.fr/account.php (ordered before May 1st 2022) or https://store.fpnet.fr/en/order-history for newer orders. You can also use the bundled updater tool (click the "Download Fab's AutoBackup 7.X" link within the program and get the updated files).

 

[Larry Sabo]

When I did the update, I got an Index out of bounds error. When I clicked cancel to close the UI, I got an address violation error. I then closed the UI without issue and tried the update again and it (AutoBackup 7 Home & Office 7.12.2.6047) said there were no more updates, so I think it's okay. Maybe it's just my system.