11/5 – 12:56:00 pm
Db integrity check is still confirmed as ongoing.
11/3 – 09:52:00 pm
After talking with engineers from our service provider, we are estimating that full mailbox functionality should be restored over the coming weekend. We realize that this is totally unacceptable, however, we will be holding our service provider to account once things are restored. Thank you for your patience, and we sincerely apologize for this ongoing issue.
11/3 – 07:46:00 pm
The database checks are still going, engineers are monitoring the progress through the read/write activity.
11/3 – 04:11:00 am
The integrity check are still continuing on both databases in the progress bar-less step. Still no numeric value at this time.
11/2 – 07:21:00 pm
Both integrity checks are still active and processing. Again no numeric value to assign as explained.
11/2 – 02:53:00 pm
Integrity chcks are moving. It’s going to be tough to give numbers as both DBs are now in progress bar less steps. Activity is confirmed.
11/2 – 12:24:00 pm
DB3 completed steps 2 and 3 out of 5, and is now on step 4. DB2 is at 72% of its final step.
11/2 – 10:15:00 am Eastern
Both DB Integrity checks are still continuing to progress.
11/2 – 08:31:00 am Eastern
Anticipating DB2 completion by end of business day. Unfortunately DB3 check is taking longer potentially completing its process by Friday.
11/2 – 08:24:00 am Eastern
Integrity checks on DB2 is on the final process at 80% while DB3 is on the second passing at 30%
11/2/2011 6:34AM Eastern
At this time integrity checks on DB2 & DB3 are progressing smoothly. We are on the final processes of DB2 while DB3 is moving flawlessly.
11:14PM Eastern
The integrity check on DB3 is on step 4/5 and at 80% DB2 is on step 5/5 and at 65%. We anticipate DB2 finishing around 4 am and DB3 late Wed
8:27 PM Eastern
The integrity checks are still progressing, the last database is through 70% of the Scanning process.
6:40 PM Eastern
The integrity checks are still progressing, the last database is through 62% of the Scanning process.
Update 3:38 PM Eastern
The integrity checks are still progressing, the last database is through 50% of the Scanning.
Update 1:09 PM Eastern
The dial tone migration has completed and users are now able to access their mailboxes on the temporary database.
Update 12:30 PM Eastern
Our service provider will be performing a dial tone migration to DEWEYMBOX2 for users on the affected databases. A dial tone migration will allow users to reconnect to their user mailbox on DEWEYMBOX2 via Outlook, OWA and Active Sync however the mailbox will have no information other than the mail from the previous day when the outage occurred and any new live running mail.
Users will see the following prompt after restarting Outlook
If the user wants to access their new mail they’ll select “Use Temporary Mailbox”
After the databases are back online they will move users back to their original databases and then restore mail from the temporary mailbox.
Update 11:14 AM Eastern The integrity checks are still progressing, the last database is through 30% of the Scanning. No ETA as there is no completion estimate.
Update 9:13 AM Eastern The Engineers have been able to successfully mount one more database and are currently monitoring the final DB integrity check.
Update 7:26 AM Eastern The larger DB is about 25% completed while the second largest DB is still checked as errors were detected.
Update 4:41 AM Eastern The larger DB check has started showing progress while the second to largest DB is still on the final step of the integrity
check.
Update 4:00 AM Eastern There hasn’t been any significant progress change as the checks are still underway.
Update 2:30 AM Eastern The integrity check is still underway. Unfortunately the first check is on the last step which does not have a progress bar which is why no update has been provided.
Update 10:30 PM Eastern The integrity check is performing clean up and is on the final step of the check. We’ve begun the check on the last DB
Update 8:55 PM Eastern The integrity check is 90% completed.
Update 8:05 PM Eastern The current DB check is 2/3 completed. At this point we estimate being able to mount the database on or before 10PM EST.
Update 7:30 PM Eastern The integrity check is about 1/3 completed. At this point we are going to wait for completion before starting the integrity check on the DB3
Update 4:55 PM Eastern The integrity check is taking slightly longer than calculated. We will continue to monitor the repairs and will update this posting once the next step is reached.
Update 3:32 PM Eastern We’ve mounted half of the databases and are now running consistency checks on the larger databases.
Update 2:59 PM Eastern The databases were offlined by Exchange and now in a dirty shutdown state. Before we can remount the databases we must run
consistency check.
Update 2:53 PM Eastern The databases for DEWEY were forcefully taken offline by Exchange. We are working on diagnosing the issue.
Update 2:16PM EasternWe’ve replaced a failed drive on the DEWEYMBOX1 server. Unfortunately to bring the server back to a healthy RAID we must rebuild the array. Throughout the rest of the work day users may see momentary delays on mail delivery if their mailbox is hosted on the affected array.
