DW Spectrum - Automatic Camera Failover

DW Spectrum® IPVMS Automatic Camera Failover

-----------------------------------

Affected Roles: Administrator, Owner

Related Digital Watchdog VMS Apps: DW Spectrum® IPVMS

Last Edit: December 7, 2021

-----------------------------------

What is Automatic Camera Failover?

When using a DW Spectrum® IPVMS System that consists of merged DW Spectrum Servers, the Automatic Camera Failover feature can be activated to allow a DW Spectrum System to automatically recover its recording status when a power or networking failure occurs with a DW Spectrum Server. Rather than all of the downed server’s associated cameras halting all recording, the Automatic Camera Failover feature tells the DW Spectrum System to divert the capture of the downed server’s cameras from the failed Server to a working that is merged within the same DW Spectrum System.

A single Server can handle a maximum of 128 cameras. So, for example, if a Server is actively recording 64 cameras it can act as a Failover Server for up to 64 additional cameras from other Servers in the System. Failover takes approximately 1 minute in the instance of a network or power failure.

This article will outline the Automatic Camera Failover feature of the DW Spectrum IPVMS platform and how to designate a server as a Failover Server and how to set Failover Priorities for devices in the DW Spectrum Client.

Defining DW Spectrum IPVMS Terms

Failover – an included feature of DW Spectrum IPVMS that allows the automatic retrieval of camera streams from DW Spectrum Servers that suddenly become unavailable for streaming or archiving video footage.
Failover Server – the DW Spectrum Server hardware that has been designated as the end-point upon rerouting camera streams from a downed server.
DW Spectrum Server – this term is used when referring specifically to the DW Spectrum software, which runs in the background of hosting hardware, or refers to the hosting hardware itself. The purpose of a DW Spectrum Server is to act as the receiving node that archives video footage, manage local camera settings, and to process and analyze video for camera motion detection and analytics.
DW Spectrum System – this term is used when referring to the DW Spectrum setup as a whole. A DW Spectrum System can be comprised of only one DW Spectrum Server or it can be comprised of several DW Spectrum Servers that have used the Merge feature to become a single system, similar to a “hive mind”, through which the servers may share resources with each other.
DW Spectrum Client – this software acts as the desktop application that provides a user interface (UI) through which users may view camera video, manage settings, and generally interact with their DW Spectrum System.

Failover Design Considerations

DW Spectrum’s server-hive architecture is flexible enough to implement the Failover feature in a variety of methods. When designing and implementing a DW Spectrum System that will utilize the Failover feature, careful consideration should be given to the systems design. A Failover Server can be designed to use the Standby Method or the Overhead Method (defined below).

For example, a DW Spectrum System is comprised of a total of 140 cameras (each using a similar video resolution), shared between three (3) merged DW Spectrum Servers.

Using this example:

Standby Method – a DW Spectrum Server, that is part of a merged system, that has no assigned cameras, and is free of most operational duties, can be assigned to act specifically as the Failover Server. Until a server failure occurs within the system, the Failover Server will standby.

Using the example above, two (2) servers are in production/recording mode, each recording 70 cameras (140 total). An additional (3rd) DW Spectrum Server has been merged into the system to act only as a Failover Server. With no cameras assigned to it and the Failover feature enabled, if either of the primary servers producing video archives happens to unexpectedly go offline (power loss, network crash, etc.), the Failover Server will begin to archive the video of cameras that were originally assigned to the downed server.

Overhead Method – DW Spectrum Servers within a merged system each are responsible for managing and archiving camera video for their assigned cameras. However, if one of the servers should unexpectedly go offline, the remaining operational servers will combine their own processing resources to fill in for the downed server.

Using the example above, three (3) servers are in production/recording mode, each recording 47 cameras in a merged system. If any of the merged servers should suddenly go offline, the overhead capacity (in processor power and storage) of the remaining 2 DW Spectrum Servers can grab the downed server’s cameras and continue to record to their own storage.

Automatic Failover with Multiple IP Subnets/Internet

The DW Spectrum Failover function tends to work best when the DW Spectrum Server, its cameras, and the Failover Server are all on the same local network (LAN) IP subnet. However, as long as the devices can communicate uninterrupted, connection is possible over WAN (Internet).

To allow an automatic failover to occur, the devices must be port forwarded to be visible to each other. Some things that can help with this:

Cameras added by Auto Discovery – configure the Failover Server to accept multicast requests from the cameras. This can be done by forwarding IP packets to multiple hosts (multicast) that are detected on the subnet of the Failover Server, using each cameras’ IP addresses as the source.
Cameras added Manually by IP address – ensure that the IP address of each camera has not changed since being added to the original server. The Failover Server will connect to these cameras by IP address by forwarding IP packets to a single recipient (unicast) on the same subnet of the Failover Server while cameras are added during manual device search and registration.

If communication for a network with multiple routers or network switches is not routed to communicate across subnets, the Automatic Camera Failover may experience issues in the event of a failover.

Please verify that the network’s routing is configured correctly to ensure a successful Automatic Camera Failover across multiple subnets. You can test communication between servers by using the “Ping” button in the General Server Settings on the original server to ping the Failover Server(s). Additionally, manually moving cameras temporarily from one server to another can also serve as a stability test.

Enabling Automatic Camera Failover for a Server:

To enable Automatic Camera Failover for a DW Spectrum Server:

Using the DW Spectrum Client, right-click on the Failover Server in the Resource Panel and select “Server Settings”.

The General Server Settings menu window will display.

Locate the Failover section, then click on the “Failover” toggle. The Failover toggle will illuminate with a green indicator when enabled.

**NOTE: By enabling the Failover toggle for a server within a merged system, you are designating that server as a Failover Server.

Find the “Max devices on this server” setting, enter the maximum number of devices that the server is allowed to take-on during a failover.

For your consideration:

- Core i3 or above CPUs – up to 128 devices (max.) supported
- ARM CPUs – up to 12 devices (max.) supported

Once finished, click the Apply button to save the settings. Click the OK button to close the window.

If you want to designate additional Failover Servers for the DW Spectrum System repeat Steps 1~3 above for each additional Failover Server.

To Setup Failover Priority:

After designating a Failover Server for the DW Spectrum System, Failover Priority settings can be applied to specify which devices will be retrieved by the Failover Server as well as the priority of the retrieval.

To set up Failover Priority:

In the DW Spectrum Client, right-click on the Failover Server in the Resource Panel and select “Server Settings”.

In the Server Settings menu, click on the “Failover Priority” button.

A list of the other servers and connected devices that are merged with the system will display. Select the cameras/devices that will be reassigned in the event of an Automatic Camera Failover. Selecting the server itself will also simultaneously select all of its registered cameras.

Select a value of High, Medium, or Low priority for the selected cameras. The higher that the set priority is, the sooner a camera failover will reassign the camera to the first available Failover Server within the DW Spectrum System.

Click the “OK” button to close the Failover Priority window, then click the “Apply” button to save the settings. Click the “OK” button in the Server Settings to close the window.

**NOTE: By default, cameras are set to a Medium priority level when a Failover Server is available. If a camera should not be reassigned during the failover process, designate the device with a “Never” priority level.