The proof of the pudding PDF Print E-mail
User Rating: / 0
Written by Michael Felt   

is in the eating!

I reported that I had suffered for a few days getting everything running properly after a power failure of about 90 minutes. I cannot place direct blame on everything that went wrong on the power outage.

To a certain degree, everything worked very well. Once power was back, everything came back up, though a bit slow because of dependencies in the different systems (e.g., better if the name server is up before the database). But I have a odd collection of processors systems - which have been great for learning - but not so great for "unmanaged" restarts.

What I learned

I learned that as convenient as it is, I am extremely on my NAS - that I am know using as all of, NFS,  iscsi, and SQL server - and when that gets confused everything gets messy. Additionally, my initial idea of having one RAID6 volume - as it gets above 40% filled - is extremely hard to protect with copies of what it has (although RAID6 itself has been excellent - two disks have failed and I could just remove that bad disk, put in a new one, and the array rebuilds itself). In other words, I decided to reformat my single RAID6 array as two RAID5 - and this meant everything had to get off.

Looking for efficiency

I started out with adding an older NAS into the network again, and using NFS to copy from one NAS to the other. But the older NAS has "only" a 100MByte Ethernet interface - and that was taking forever - more time than I had. So I went back to the newer NAS and starting creating large tarballs of individual directories - and when these were finished used ftp to copy them to my desktop at 1GBit speeds. I wrote about the the nearly 400GByte (or was it 780GB) transfer that took nearly 4 hours - that would have been two days without this improvement.

Saved by application redundancy

Unfortunately, I lost on large project directory - all the gnu packages I have packaged over the years. Mainly the little notes. How? Well, officially - user-error - as much as I would like to blame my environment for being too hectic (go cut the grass! Are you planning on drinking coffee with me or not? - in other words - times that I need to get away from the keyboard and be part of a family (in fact it was my children's birthdays - two of them a week apart - not household chores I could ignore a bit longer :) )

And while I remember to remake a copy of the tar file of the gnu projects - I neglected to copy it to my desktop system (and two other tar files, more in a moment) so when I reformated the RAID6 - I realized about 15 seconds too late that I had not copied those files - a second time - but had unfortunately already destroyed the stale copy I had meant to replace. Sigh.

The other two tar files. Well, it turns out this project directory could be reconstructed from other backups and source-control files that I did have elsewhere. And those would have been much more painful to lose because they are all "original" work - unlike the gnu opensource that I can just download again.

Time to eat pudding

So, now it is time to eat my pudding - named buildaix - and see how hard/easy it will be to rebuild new packaging (installp) packages. I was intending to all along as I developed the buildaix-2.0.1 version - and know I must. So, while I would have preferred to not have to admit I made a user error I am fortunate that I only lost something that is supposed to be "easy", even if time-consuming, to replace.

< Prev   Next >