turnsout.com

View Original

Well this day went from ok to absolute suck

So I came in today, looked at email, read my news, went to my 10:AM meeting and then come back to my desk to settle in for a long day of programming. I start to get comfortable, and then I hear Brandon (co-worker same job title as me) and Doug (Server support) talking about a maybe failed drive on our test server (where we have been developing benefits).

So I quick copy the scripts and back up the databases off of that server and we start to work on rectifying this situation.

So there are two raids on that server, one that has two drives another that has 3. On the mirrored drives one of the drives failed (no big deal, just gotta replace, right?). On the Raid 5 (3 drives), one drive had totally failed and one was giving an about to fail message (PFA, I think). Now that'd be bad, cause that's pretty much unrecoverable from.

So now we're hoping that the Raid 5 will still be able to use the about to fail drive to recover the failed drive and then replace the about to fail drive.

So the first drive recovery should be done now, so I'm gonna head back and see what Doug and Nathan (another server support guy) have to say about it.

Yeah we have backups, but working on our slow (but not busy) test server is a lot better then working on our intranet server that is slow (4 years old) and very busy (avg utilization between 40-60%).

I can't wait to get new hardware. We will order it soon and then we can retire our 3 servers.