Planning Ahead

With the number of threats to data security growing, disaster recovery (DR) planning should be a top priority for businesses to ensure rapid recovery and minimal downtime. Yet, many companies do not even have a plan in place and, of those that do, most are not testing or managing those plans to industry standards.

With data security being a top concern for most organizations, TeraGo partnered with IDC Canada to conduct an evaluation of Canadian businesses and their responses to DR processes.

The survey involved more than 200 Canadian companies representing 20-plus industry aggregations, including business/professional services, manufacturing, financial services, government, retail, communications, healthcare and utilities. No single industry aggregate represented more than 13% of the survey base.

The study found that polled Canadian businesses are not prioritizing DR, and of those that do have a strategy in place, 81% are not testing them to industry standards. For example, ISO/IEC 27031, from the International Organization for Standardization and the International Electrotechnical Commission, is the primary international standard for DR and business continuity of IT and communications systems.

COST-BENEFIT ANALYSIS OF DR PLANNING

The Federal Emergency Management Agency (FEMA) in the United States reports that more than 40% of businesses do not reopen after a disaster, and another 25% fail within one year. Not having or properly testing a DR plan can have critical fallout, including likely resulting in financial loss from the following:

• Downtime: Inefficient DR planning leaves businesses at risk of losing mission-critical data and can cost a company hundreds of thousands of dollars per hour of downtime. This could include confidential financial data or sensitive customer data, the loss of which could potentially have an impact on brand reputation as well.
• Re-allocation of resources: Beyond losing access to data systems and the ability to transact business, resources are diverted from normal business operations to manage the crisis situation.
• Damage to reputation: Businesses that suffer from a disaster – especially one that involves a data breach – often see a drop in both customer and employee confidence.
• Impacted stock price: For publicly traded companies, damage to the brand can result in stock prices being affected.
• Legal issues: Non-compliance by those in the insurance industry could have significant implications as a result of actions by the Canadian Council of Insurance Regulators and provincial or federal insurance regulatory bodies.

With the volume of data growing every year, now more than ever, it is critical for companies of all types to implement a robust DR plan.

EFFECTIVE DR PLANNING

There are eight essential steps to building, implementing and managing a DR plan.

1     DR planning begins with a comprehensive assessment of the threats and dependencies that could have an impact on business operations and data security, including the following different types of failures.

• Operational: Often a company’s IT infrastructure is not equipped to handle the heavy workloads under which it is used. Put under major stress, systems can exceed back-up battery capacity, resulting in circuit breaker failure and breakdown of other IT equipment.
• Human-induced: Internally, careless, uninformed or disgruntled employees can cause harm to the security and integrity of a company’s data, while hacking, terrorism and vandalism from outside sources are also very real threats that companies are facing today.
• Natural disasters: Unpredictable environmental catastrophes, such as floods, extreme temperatures and power outages, can interrupt operations for hours, days or even weeks before regular business can resume.
• Dependencies: Third-party partners, suppliers and service providers that face a disaster can have just as much of an impact on a business’ operations.

2     Conduct a risk assessment and a business impact analysis (BIA) to fully understand what IT services are necessary to support the company’s critical business activities.

3     Define the recovery time objective (RTO), the amount of time a company can effectively operate with systems down, and recovery point objective (RPO), a company’s loss tolerance to data, for all critical applications. RTOs and RPOs both play a critical part in creating a comprehensive BIA for the DR plan.

4     Identify key infrastructure and assess gaps, especially for mission-critical applications, and prioritize their failover, as well as plan for duplication of critical skills.

5     Define policies and establish which tools are necessary to have on site, off site or with a vendor that can validate the outlined DR procedures.

6     Develop an easy-to-use, repeatable process that covers off each step for recovering damaged IT assets and clearly outlines the procedures necessary to recover them and return to their normal operation as soon as possible.

7     Test frequently and simulate various disasters, implementing

the plan for all contingencies, including the training of relevant staff members on the processes and procedures in DR scenarios, and outlining who does what, when and how.

8     Document time-to-remediation for all elements of IT infrastructure so that the potential impact of downtime can be mitigated at all times.

TESTING THE DR PLAN

Just as important as having a DR plan is testing it regularly to determine its efficiency and effectiveness. The aforementioned study found that 81% of polled Canadian businesses are not testing their DR plan to industry standards.

Among ISO/IEC 27031’s testing objectives is to build confidence throughout the organization that the DR plan will satisfy business requirements; demonstrate that the critical systems can be recovered/restored to agreed service levels; provide staff with an opportunity to exercise the DR plan and its execution, including hands-on training; and verify that DR plans and the DR environment are properly synchronized with the production environment and the business.

In line with the aforementioned testing objectives, there are three recommended approaches to testing:

• Walkthrough: Individuals involved in the DR planning have an initial walkthrough meeting to discuss objectives, roles, responsibilities and dependencies to ensure that the process is fully understood.
• Simulated recovery – system subset test: A critical subset of the environment is scoped and test performed (for example, failover and failback of critical servers and network components) to ensure the DR plan is effective for the subset.
• Operational test: This is a wider-scale recovery test that is inherently more complex and introduces additional risk to the normal functioning of systems running the business.

The test scenarios should be exercised at different intervals and are ideally introduced randomly to obtain a more accurate sense of the organization’s state of readiness and preparedness. Industry best practice defines regular testing as quarterly, though walkthroughs should be done whenever staff changes.

It is usually sufficient to conduct operational tests once a year, depending on the organization’s risk profile.

DR TESTING CHALLENGES AND SOLUTIONS

Regular DR testing requires a significant amount of resources, which many companies are hesitant to commit. Not only is there a time commitment, there is the logistical cost of organizing and executing DR testing, as well as the productivity cost from diverting staff time and effort away from other priority projects.

A lack of human resources is often where organizations fall short. In fact, 36% of surveyed businesses admitted that they do not have enough qualified staff to implement a DR plan successfully.

To overcome these challenges, organizations should first define their risk profile by conducting a full audit and cost-benefit analysis. This will determine the organization’s risk appetite and the most effective and efficient plan based on that assessment.

Next, breaking down the testing will eliminate the difficulty of testing the whole set of DR plan elements and processes in one test exercise. Finally, consider using a managed service provider to outsource the test to a third party, thereby allowing the core business team to concentrate on operating the business while the provider deals with any technical difficulties.

DR PLANNING FOR THE INSURANCE INDUSTRY

For insurance companies that are in the business of helping people through disasters, having a robust DR plan is paramount. Insurance companies have an obligation to their clients to be available when disaster strikes, so they must be fully functioning regardless of extenuating circumstances.

The implications of downed systems, data loss or being unable to service customers in their time of need could not only damage their reputations, it could also impede their responses to policyholders and the ability to satisfy regulatory requirements of provincial and federal regulatory bodies.

A robust DR strategy, though perhaps not explicitly regulated, is a critical adjunct to an insurance company’s overall risk management and governance strategy.

THE BOTTOM LINE

Disaster recovery planning is critical in this globally connected environment. Canadian companies need a robust data recovery plan to protect data critical to their business operations, including their customer’s private personal data, or risk financial loss.

When developing a DR strategy that will hold up against internal and external threats, a company needs to consider budgets, senior management’s tolerance to risk and industry-specific regulatory obligations.

An advisor can help to strategize, develop, test, manage and execute the plan, while also assisting in minimizing business disruption.