Disaster Recovery / Continuity of Operations

Initiatives / Accomplishments in Place

For any government organization it’s important for data to be continually secure and available. Numerous events over the past few years have led to a heightened awareness of the need to be prepared for the worst. IT departments are now faced with the pressing concerns of growing demand for and reliance on access to information, along with ensuring continuous operation during a disaster. The City of Roseville is always working to be proactive to insure we handle any situation thrown our way to the best of our abilities.

One of the ways the City has prepared is through the Business Impact Analysis (BIA). The BIA is an initial step in creating a comprehensive Business Resumption Plan. It differentiates critical (urgent) and non-critical (non-urgent) organization functions/activities.

The City partnered with MLC to develop a high level Business Impact Analysis plan. The deliverable from this engagement will be the ground work for more detailed business resumption and disaster recovery plans for those identified critical business processes / services.

This project is a prime example of the importance of our partnership with Business and Technology. We will work together to identify these key business processes / services. This will enable the City’s next steps to ensure we have a plan in place that staff can execute and ensure the City continues to conduct its critical business in time of emergency or disaster.

In addition to this BIA plan, the City of Roseville continues to conduct citywide Emergency Operations Exercise. This last year focused on a cyber-related incident. This first of its kind exercise, involved every department in the city, as well as other regional partners. A fictitious cyber breach incident was used to mock outages of critical infrastructure including the finance system and the city’s power plant. The exercise was extremely successful in raising awareness among city staff on how a cyber incident can have widespread impacts on the City’s critical infrastructure and even threaten public safety and well-being. As part of this effort, the city’s Emergency Operations Team developed a Cyber Annex to the city’s overall Emergency Operations Plan. This document details actions, policies and responsibilities to follow in response to a cyber related incident. The city is now more prepared than ever to respond to a critical cyber incident.

Initiatives / Accomplishments in the Last Year

Business Impact Analysis & Business Continuity Strategies

The City proactively leveraged the work from the initial Business Impact Analysis by creating detailed Business Continuity Strategies for 5 of the City’s most critical processes.

The process selected:
Utilities: Waste Water (Collections) Manage and Operate Waste Water Collections A01
Public Works Administration Emergency Repair and operations of the City’s critical infrastructure A0
Information Technology: Operations Monitor, Manage, and Maintain the City Network Infrastructure A0
Police Operations Answer Incoming 911 Calls and Relay Information to Dispatch A0
Utilities: Electric Provide Electric Service Delivery to Customers A01

The Business Continuity Strategy Sessions covered:

  • Review the critical business processes
  • Review the critical dependencies
  • Develop a high level workflow of the selected process
  • Develop alternate strategies for the selected process
  • Key Personnel - Primary, Secondary, Alternate
  • Internal Departments – Strategies to work with other departments and/or what to do if another department is unavailable
  • External Vendors – Strategies for working with critical vendors during an event as well as alternate vendors if the primary is unavailable
  • IT Dependencies and workarounds in terms of applications, data, infrastructure, connectivity / communication, data feeds, reports, etc.
  • Other Requirements (may include physical location, tools, equipment, supplies, forms, etc.)
  • The results we detailed plans that the City can put to active use not only in emergency but in our EOC exercise and cross training efforts.

    Roseville Emergency Operations Center Upgrade, Cyber EOC Exercise & Cyber Annex

    After a needs assessment was conducted it was identified that there is a significant technology gap with available EOC resources. This project upgrades the technology used in the EOC for display of that information.

    Major changes made to the EOC consist of:

  • Three new flat panel displays. The addition of one display will provide the ability to display additional incident information.
  • Display capabilities will be added to nearby meeting rooms – also used as part of the EOC.
  • Two mounted projectors plus up to five displays on mobile carts will be added.
  • The usability of the system controls will be improved and accessible through an iOS or Android app.

    Status of City Initiatives (next 12 - 18 months)

    IX-3. Select the status for each city item/ area below:
    No Plans to Use/ Deploy
    No Plans, but Considering/ Under Discussion
    Definite Plans to Procure/ Deploy/ Upgrade in next 12-18 mos.
    In Use Now
    IX-3.1 Redundant/ Offsite Data Storage        X
    IX-3.2 Cyber disruptions are included in city’s Crisis / Emergency Management Plan        X
    IX-3.3 Post-disaster Plans for Use of Social Media    X    
    IX-3.4 Volunteer Disaster Registries (nurses, EMTs, trained citizens)    X    
    IX-3.5 Next-Generation 9-1-1        X
    IX-3.6 Integrated Video Surveillance    X    
    IX-3.7 Other (describe below)        

    In 2016 the City of Roseville conducted an Emergency Operations “Cyber” Exercise. As part of the planning effort the City developed a Cyber Annex to address technological and cyber threats as defined in the City’s 2016 Multi-Hazard Mitigation Plan. The threat of a Cyber-attack is more realistic than ever before. While recognizing the importance of technology in all business processes the City realizes the importance of preparing for these kinds of attacks.

    With the completion of the recent Business Impact Analysis (BIA) the City is evaluating the next steps of business continuity. The BIA prioritized processes and developed 4 draft “business resumption plans”. The City is now going to use those as a template for creating “business resumption plans” for the remaining business processes. This will be a multi-year project involving all the departments of the organization. In some cases will depend on consultants to provide support or guidance on the “business resumption plans”. (Strategic Planning & Digital Communications Collaboration Site Report: Analysis Report)