--- canonical: https://safekit.evidian.com/wp-content/uploads/downloads_safekit/version-82/safekituserguidehtml/documentation/safekituserguideen.htm --- Evidian SafeKit User's Guide 8.2 **SafeKit User's Guide** High Availability Software for Critical Applications **Overview** | | | | | --- | --- | --- | | **Subject** | This document covers all the phases of the SafeKit implementation: architecture, installation, tests, administration & troubleshooting, support, and command line interface. | | | **Intended Readers** | **Architectures** | Technical overview | | **Installation** | Installation | | **Console** | The SafeKit web console Securing the SafeKit web service | | **Advanced configuration** | Cluster.xml for the SafeKit cluster configuration Userconfig.xml for an application module configuration Scripts for an application module configuration Examples of application module configurations | | **Administration** | Mirror module administration Farm module administration Command line interface Advanced administration | | **Support** | Tests Troubleshooting SafeKit support Log Messages Index | | **Other** | Table of Contents Third-Party Software | | Release | SafeKit 8.2 | | | **Supported OS** | Windows and Linux; for a detailed list of supported OS, see here | | | **Web Sites** | Marketing site               Support portal | | | Ref | 39 A2 38MC 07 | | | | | | Copyright © Eviden, 2026 The trademarks mentioned in this document are the propriety of their respective owners. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical or otherwise without the prior written permission of the publisher. Eviden disclaims the implied warranties of merchantability and fitness for a particular purpose and makes no express warranties except as may be stated in its written agreement with and for its customer. In no event is Eviden liable to anyone for any indirect, special, or consequential damages. The information and specifications in this document are subject to change without notice. Consult your Eviden Marketing Representative for product or service availability. **Table of Contents** Overview.. 3 Table of Contents. 5 1.... Technical overview.. 17 1.1..... Generalities, solutions, architectures 17 1.1.1       Introduction to SafeKit 17 1.1.2       SafeKit solutions. 17 1.1.3       SafeKit architectures. 18 1.1.4       SafeKit cluster definition. 18 1.1.5       SafeKit application module definition. 19 1.1.6       SafeKit limitations. 19 1.2..... The SafeKit mirror cluster 20 1.2.1       Real time file replication and application failover 20 1.2.2       Step 1. Normal operation. 21 1.2.3       Step 2. Failover 21 1.2.4       Step 3. Failback and automatic resynchronization. 21 1.2.5       Step 4. Return to normal operation. 22 1.2.6       Synchronous replication versus asynchronous replication. 22 1.2.7       Behavior in case of network isolation. 22 1.2.8       3-node replication. 23 1.2.9       SafeKit on a single node to protect against software failures 23 1.3..... The SafeKit farm cluster 24 1.3.1       Network load balancing and application failover 24 1.3.2       Principle of a virtual IP address with network load balancing. 24 1.3.3       Load balancing for stateful or stateless web services 24 1.3.4       Chain high availability solution in a farm.. 25 1.4..... Clusters running several modules 25 1.4.1       The SafeKit farm+mirror cluster 25 1.4.2       The SafeKit active/active cluster with replication. 25 1.4.3       The SafeKit N-1 cluster 26 1.5..... The SafeKit Hyper-V or KVM cluster 27 1.5.1       Load balancing, replication, failover of entire virtual machines. 27 1.6..... SafeKit clusters in the cloud. 27 1.6.1       Mirror cluster in Azure, AWS and GCP. 27 1.6.2       Farm cluster in Azure, AWS and GCP. 28 2.... Installation. 31 2.1..... SafeKit install 31 2.1.1       Download the package. 31 2.1.2       Installation directories and disk space provisioning. 32 2.1.3       SafeKit install procedure. 32 2.1.4       Use the SafeKit web console or command line interface. 35 2.1.5       SafeKit license keys. 37 2.1.6       System specific procedures and characteristics 37 2.2..... Mirror installation recommendation. 38 2.2.1       Hardware and system prerequisites 38 2.2.2       Network prerequisites. 38 2.2.3       Application prerequisites. 38 2.2.4       File replication prerequisites. 38 2.3..... Farm installation recommendation. 39 2.3.1       Hardware and system prerequisites 39 2.3.2       Network prerequisites. 39 2.3.3       Application prerequisites. 39 2.4..... SafeKit upgrade. 39 2.4.1       Prepare the upgrade. 39 2.4.2       Uninstall procedure. 40 2.4.3       Reinstall and postinstall procedure. 40 2.5..... SafeKit full uninstall 42 2.5.1       Uninstall on Windows as administrator 42 2.5.2       Uninstall on Linux as root 43 2.6..... SafeKit documentation. 44 3.... The SafeKit web console. 45 3.1..... Start the web console. 45 3.1.1       Start a web browser 45 3.1.2       Connect to a SafeKit node. 45 3.1.3       List of connection nodes 46 3.1.4       Use the SafeKit web application. 47 3.1.5       Update the web console. 49 3.2..... Configure the cluster 49 3.2.1       Cluster configuration wizard. 50 3.2.2       Cluster configuration home page. 53 3.3..... Configure an application module. 54 3.3.1       Select the new module to configure. 55 3.3.2       Module configuration wizard. 56 3.3.3       Modules configuration home page. 61 3.3.4       Edit the module configuration locally and then apply it 63 3.4..... Monitor an application module. 64 3.4.1       Monitoring home page. 64 3.4.2       Module state. 65 3.4.3       Module control menus 67 3.4.4       Module details. 70 3.4.5       Module states timeline. 75 3.5..... Snapshots or logs of application module for debug and support 76 3.6..... Secure access to the web console. 77 4.... Tests. 79 4.1..... Installation and tests after boot 79 4.1.1       Test package installation. 79 4.1.2       Test license and version. 80 4.1.3       Test SafeKit services and modules after boot 80 4.1.4       Test start of SafeKit web console. 82 4.2..... Tests of a mirror module. 83 4.2.1       Test first start of a mirror module on 2 servers ![](safekituserguideen_fichiers/image002.png)STOP (NotReady). 83 4.2.2       Test start of a mirror module on 2 servers ![](safekituserguideen_fichiers/image002.png)STOP (NotReady). 83 4.2.3       Test stop of a mirror module on the server ![](safekituserguideen_fichiers/image003.png)PRIM (Ready). 83 4.2.4       Test start of a mirror module on the server ![](safekituserguideen_fichiers/image002.png)STOP (NotReady). 84 4.2.5       Test restart of a mirror module on the server ![](safekituserguideen_fichiers/image004.png)PRIM (Ready). 84 4.2.6       Test virtual IP address of a mirror module. 84 4.2.7       Test file replication of a mirror module. 85 4.2.8       Test shutdown of the server ![](safekituserguideen_fichiers/image003.png)PRIM (Ready). 86 4.2.9       Test power-off of the server ![](safekituserguideen_fichiers/image004.png)PRIM (Ready). 87 4.2.10      Test split-brain with a mirror module. 87 4.2.11      Continue your mirror module tests with checkers 89 4.3..... Tests of a farm module. 89 4.3.1       Test start of a farm module on all servers ![](safekituserguideen_fichiers/image002.png)STOP (NotReady). 89 4.3.2       Test stop of a farm module on one server ![](safekituserguideen_fichiers/image003.png)UP (Ready). 89 4.3.3       Test restart of a farm module on one server ![](safekituserguideen_fichiers/image005.jpg)UP(Ready). 89 4.3.4       Test virtual IP address of a farm module. 89 4.3.5       Test TCP load balancing on a virtual IP address. 91 4.3.6       Test split-brain with a farm module. 92 4.3.7       Test compatibility of the network with invisible MAC address (vmac\_invisible) 93 4.3.8       Test shutdown of a server ![](safekituserguideen_fichiers/image005.jpg)UP (Ready). 94 4.3.9       Test power-off of a server ![](safekituserguideen_fichiers/image005.jpg)UP (Ready). 95 4.3.10      Continue your farm module tests with checkers. 95 4.4..... Tests of checkers common to mirror and farm.. 95 4.4.1       Test checker with action restart or stopstart 95 4.4.2       Test checker with action restart or stopstart 96 4.4.3       Test checker with action wait 97 4.4.4       Test with action wait 98 4.4.5       Test checker with action wait 99 4.4.6       Test checker with action wait 99 4.4.7       Test checker with action wait 100 4.4.8       Test checker with action restart or stopstart 101 5.... Mirror module administration. 105 5.1..... Operating mode of a mirror module. 105 5.2..... State automaton of a mirror module (STOP, WAIT, ALONE, PRIM, SECOND - NotReady, Transient, Ready) 107 5.3..... First start-up of a mirror module (safekit prim command) 108 5.4..... Different reintegration cases (use of bitmaps) 109 5.5..... Start-up of a mirror module with the up-to-date data  ![](safekituserguideen_fichiers/image006.jpg)STOP (NotReady) - ![](safekituserguideen_fichiers/image007.png)WAIT (NotReady). 110 5.6..... Degraded replication mode (![](safekituserguideen_fichiers/image003.png)ALONE (Ready) degraded) 111 5.7..... Automatic or manual failover 112 5.8..... Default primary server (automatic swap after reintegration) 114 5.9..... Prim command fails: why? (safekit primforce command) 115 6.... Farm module administration. 117 6.1..... Operating mode of a farm module. 117 6.2..... State automaton of a farm module (STOP, WAIT, UP - NotReady, Transient, Ready) 118 6.3..... Start-up of a farm module. 119 7.... Troubleshooting. 121 7.1..... Connection issues with the web console. 121 7.1.1       Browser check. 122 7.1.2       Browser state clear 122 7.1.3       Server check. 122 7.2..... Connection issues with the HTTPS web console. 123 7.2.1       Check server certificates. 123 7.2.2       Check certificates installed in SafeKit 124 7.2.3       Revert to HTTP configuration. 125 7.3..... Global environment checks (healthcheck script) 125 7.4..... How to read logs and resources of the module? 125 7.5..... How to read the commands log of the server? 126 7.6..... Stable module  ![](safekituserguideen_fichiers/image003.png)(Ready) and ![](safekituserguideen_fichiers/image003.png)(Ready). 126 7.7..... Degraded module ![](safekituserguideen_fichiers/image003.png)(Ready)and ![](safekituserguideen_fichiers/image008.png)/![](safekituserguideen_fichiers/image007.png)(NotReady). 126 7.8..... Out of service module ![](safekituserguideen_fichiers/image008.png)/![](safekituserguideen_fichiers/image007.png)(NotReady) and ![](safekituserguideen_fichiers/image008.png)/![](safekituserguideen_fichiers/image007.png)(NotReady). 127 7.9..... Module ![](safekituserguideen_fichiers/image008.png) STOP (NotReady): start the module. 127 7.10... Module ![](safekituserguideen_fichiers/image007.png)WAIT (NotReady): repair the resource="down" 127 7.11... Module oscillating from ![](safekituserguideen_fichiers/image003.png) (Ready) to ![](safekituserguideen_fichiers/image009.png) (Transient). 128 7.12... Message on stop after maxloop. 129 7.13... Module ![](safekituserguideen_fichiers/image003.png) (Ready) but non-operational application. 130 7.14... Mirror module ![](safekituserguideen_fichiers/image003.png)ALONE (Ready) - ![](safekituserguideen_fichiers/image007.png)WAIT/![](safekituserguideen_fichiers/image008.png)STOP (NotReady) 130 7.15... Farm module ![](safekituserguideen_fichiers/image003.png)UP(Ready)but problem of load balancing in a farm.. 131 7.15.1      Reported network load share are not coherent 131 7.15.2      virtual IP address does not respond properly. 131 7.16... Problem with the virtual IP after failover 132 7.17... Problem after Boot 133 7.18... Analysis from snapshots of the module. 133 7.18.1      Module configuration files 134 7.18.2      Module dump files. 135 7.19... Problem with the size of SafeKit databases 138 7.20... Problem for retrieving the certification authority certificate from an external PKI 139 7.20.1      Export CA certificate(s) from public certificates 139 7.21... Issue with email sending by the SafeKit notification agent 141 7.21.1      Failed to read or parse the configuration file. 142 7.21.2      Email sending test blocked. 142 7.21.3      Curl errors 142 7.22... Issue with antivirus 144 7.23... Issue with SafeKit kernel modules 144 7.23.1      Mirror module with rfs filter in Windows. 144 7.23.2      Farm module with vip kernel module in Linux. 144 7.24... Troubleshooting VIP ↔ MAC resolution. 145 7.24.1      Check cache entries. 145 7.24.2      Force a new resolution. 145 7.24.3      Observe Resolution Requests. 145 7.25... Still in trouble. 146 8.... SafeKit support 147 9.... Command line interface. 149 9.1..... Commands to control and setup SafeKit 149 9.1.1       safeadmin service. 149 9.1.2       safewebserver service. 150 9.1.3       Email notification agent 151 9.1.4       SNMP service. 152 9.2..... Command lines to configure and monitor the cluster 152 9.3..... Command lines to control application modules 154 9.4..... Command lines to monitor application modules 156 9.5..... Command lines to configure application modules 157 9.6..... Command lines for support 159 9.6.1       Application module log. 159 9.6.2       Application module snapshot 159 9.6.3       Other commands 162 9.7..... Command lines during the maintenance of the module application. 162 9.7.1       Application module control for maintenance. 163 9.7.2       Running the application without the module. 164 9.8..... Command lines distributed across multiple SafeKit servers 164 9.9..... Examples 166 9.9.1       Local and distributed command. 166 9.9.2       Cluster configuration with command line. 166 9.9.3       Application module configuration with command line. 166 9.9.4       Application module snapshot with command line. 167 10. Advanced administration and setup. 169 10.1... SafeKit environment variables and directories 169 10.1.1      Global 169 10.1.2      Application module. 169 10.2... SafeKit services and daemons 171 10.2.1      SafeKit services 171 10.2.2      SafeKit daemons per module. 172 10.3... Firewall settings 172 10.3.1      Firewall settings in Linux. 173 10.3.2      Firewall settings in Windows. 174 10.3.3      Other firewalls. 174 10.4... Boot and shutdown setup in Windows 177 10.4.1      Automatic procedure. 178 10.4.2      Manual procedure. 178 10.5... Linux Secure boot settings for SafeKit kernel modules 178 10.6... Antivirus settings 179 10.7... Encryption of application module communications 180 10.7.1      Configuration with the SafeKit Web console. 181 10.7.2      Configuration with the Command Line Interface. 181 10.7.3      Advanced configuration. 181 10.8... Encryption of sensitive files in SafeKit 183 10.9... SafeKit web service settings 183 10.9.1      Configuration files 184 10.9.2      Connection ports configuration. 185 10.9.3      HTTP/HTTPS and user authentication configuration. 186 10.9.4      SafeKit API 186 10.10. SafeKit email notification agent 188 10.10.1    SafeKit notification agent configuration. 189 10.10.2    SMTP client credentials setup for authentication. 190 10.10.3    Email sending test 191 10.10.4    SafeKit notification agent activation. 191 10.11. SNMP monitoring. 192 10.11.1    SNMP monitoring in Windows 192 10.11.2    SNMP monitoring in Linux. 193 10.11.3    The SafeKit MIB. 193 10.12. Commands log of the SafeKit server 194 10.13. SafeKit log messages in system log. 194 11. Securing the SafeKit web service. 197 11.1... Overview. 197 11.1.1      Default setup. 198 11.1.2      Predefined setups 198 11.2... HTTP setup. 199 11.2.1      Default setup. 199 11.2.2      Unsecure setup based on identical role for all 201 11.3... HTTPS setup. 202 11.3.1      HTTPS setup using the SafeKit PKI 203 11.3.2      HTTPS setup using an external PKI 211 11.4... User authentication setup. 215 11.4.1      File-based authentication setup. 215 11.4.2      LDAP/AD authentication setup. 218 11.4.3      OpenID authentication setup. 220 12. Cluster.xml for the SafeKit cluster configuration. 223 12.1... Cluster.xml file. 223 12.1.1      Cluster.xml example. 223 12.1.2      Cluster.xml syntax. 224 12.1.3      , , attributes. 224 12.2... SafeKit cluster Configuration. 226 12.2.1      Configuration with the SafeKit web console. 226 12.2.2      Configuration with command line. 227 12.2.3      Configuration changes. 227 13. Userconfig.xml for an application module configuration. 229 13.1... Time-based attributes 230 13.1.1      Time-based attribute example. 230 13.1.2      Time-based attribute syntax. 230 13.2... Macros - . 231 13.2.1      example. 231 13.2.2      syntax. 231 13.2.3      attributes 231 13.3... Farm or mirror module - . 232 13.3.1      example. 232 13.3.2      syntax. 232 13.3.3      attributes 233 13.4... Heartbeats - , . 235 13.4.1      example. 236 13.4.2      syntax. 236 13.4.3      , attributes. 236 13.5... Farm topology - , . 238 13.5.1      example. 238 13.5.2      syntax. 239 13.5.3      , attributes 239 13.6... Virtual IP address - . 240 13.6.1      example in a mirror module. 240 13.6.2      example in a farm module. 241 13.6.3      Alternative to . 241 13.6.4      syntax. 242 13.6.5      , , , , attributes. 243 13.6.6      , , , attributes. 247 13.6.7      Load balancing description. 248 13.7... File replication - , . 250 13.7.1      example. 250 13.7.2      syntax. 251 13.7.3      , attributes 252 13.7.4      description. 260 13.8... Module scripts - , . 269 13.8.1      example. 269 13.8.2      syntax. 269 13.8.3      , attributes 269 13.9... Virtual hostname - , . 270 13.9.1      example. 270 13.9.2      syntax. 270 13.9.3      , attributes 271 13.9.4      description. 271 13.10. Process or service monitoring - , . 272 13.10.1    example. 272 13.10.2    syntax. 273 13.10.3    , attributes 274 13.10.4    commands 278 13.11. Checkers - . 280 13.11.1    example. 280 13.11.2    syntax. 280 13.11.3    description. 281 13.12. TCP checker - . 284 13.12.1    example. 284 13.12.2    syntax. 284 13.12.3    attributes. 285 13.13. Ping checker - . 286 13.13.1    example. 287 13.13.2    syntax. 287 13.13.3    attributes. 287 13.14. Interface checker - . 289 13.14.1    example. 289 13.14.2    syntax. 290 13.14.3    attributes. 290 13.15. IP checker - . 290 13.15.1    example. 290 13.15.2    syntax. 291 13.15.3    attributes. 291 13.16. Custom checker - . 292 13.16.1    example. 292 13.16.2    syntax. 293 13.16.3    attributes. 293 13.17. Module checker - . 295 13.17.1    example. 295 13.17.2    syntax. 295 13.17.3    attributes 296 13.18. Splitbrain checker - . 297 13.18.1    example. 297 13.18.2    syntax. 298 13.18.3    attributes 298 13.19. Failover machine - . 299 13.19.1    example. 299 13.19.2    syntax. 300 13.19.3    attributes. 300 13.19.4    description. 301 14. Scripts for an application module configuration. 305 14.1... List of scripts 305 14.1.1      Start/stop scripts. 305 14.1.2      Other scripts. 307 14.2... Variables and arguments passed to scripts 307 14.3... Scripts output 308 14.3.1      Output into script log. 308 14.3.2      Output into module log. 308 14.4... Scripts execution automaton. 309 14.5... SafeKit special commands for scripts 310 14.5.1      Commands for Windows. 311 14.5.2      Commands for Linux. 311 14.5.3      Commands for Windows and Linux. 312 15. Examples of application module configurations. 315 15.1... Mirror module example with mirror.safe. 316 15.1.1      Cluster configuration with two networks. 316 15.1.2      Mirror module configurations. 317 15.1.3      Mirror module scripts. 319 15.2... Farm module example with farm.safe. 321 15.2.1      Cluster configuration with three nodes 321 15.2.2      Farm module configurations 322 15.2.3      Farm module scripts 329 15.3... Macro and script variables example with hyperv.safe. 332 15.3.1      Module configuration with macros and var 332 15.3.2      Module scripts with var 333 15.4... Process monitoring example with softerrd.safe. 334 15.4.1      Module configuration with process monitoring. 334 15.4.2      Advanced configuration of module scripts. 335 15.5... TCP checker example. 337 15.6... Ping checker example. 338 15.7... Custom checker example with customchecker.safe. 340 15.7.1      Module configuration with custom checker 340 15.7.2      Advanced configuration of module checker script 342 15.8... Split-brain checker example. 343 15.9... Module checker examples 344 15.9.1      Example of a farm module depending on a mirror module. 344 15.9.2      Example with leader.safe and follower.safe. 346 15.10. Interface checker example. 346 15.11. IP checker example. 347 15.12. Virtual hostname example with vhost.safe. 348 15.12.1    Module configuration with a virtual hostname. 348 15.12.2    Module scripts with a virtual hostname. 349 16. SafeKit cluster in the cloud. 353 16.1... SafeKit cluster in Amazon AWS. 353 16.1.1      Mirror cluster in AWS. 354 16.1.2      Farm cluster in AWS. 355 16.2... SafeKit cluster in Microsoft Azure. 356 16.2.1      Mirror cluster in Azure. 357 16.2.2      Farm cluster in Azure. 359 16.3... SafeKit cluster in Google GCP. 360 16.3.1      Mirror cluster in GCP. 361 16.3.2      Farm cluster in GCP. 362 17. Third-Party Software. 365 Log Messages Index. 369 Index. 373 # 1. Technical overview ![*](safekituserguideen_fichiers/image001.png)       Section 1.1 “Generalities, solutions, architectures” ![*](safekituserguideen_fichiers/image001.png)       Section 1.2 “The SafeKit mirror cluster” ![*](safekituserguideen_fichiers/image001.png)       Section 1.3 “The SafeKit farm cluster” ![*](safekituserguideen_fichiers/image001.png)       Section 1.4 “Clusters running several modules” ![*](safekituserguideen_fichiers/image001.png)       Section 1.5 “The SafeKit Hyper-V or KVM cluster” Section 1.6 “A free trial of the KVM cluster with SafeKit is available here. ![*](safekituserguideen_fichiers/image001.png)       SafeKit clusters in the cloud” ## 1.1             Generalities, solutions, architectures ### 1.1.1         Introduction to SafeKit SafeKit is a high availability software solution designed to ensure 24/7 uptime for business-critical applications. It supports both Windows and Linux platforms and eliminates the need for shared disks, enterprise editions of databases, or advanced technical skills, making it a cost-effective alternative to traditional clustering solutions. Key Features: ·         Real-Time Synchronous Replication: Continuous data replication across nodes to prevent data loss. ·         Automatic Failover and Failback: Seamless switch to a secondary system during failures and reversion once the original system is operational. ·         Load Balancing: Optimizes resource use by distributing workloads across multiple servers. ·         Platform Agnostic: Compatible with physical machines, virtual machines, and public cloud infrastructures. Key Advantages: ·         Zero Specific Skills: No specialized IT skills required for deployment. ·         Zero Hardware Overhead: No need for specific hardware like shared disks or load balancers. ·         Zero Software Overhead: Works with standard editions of Windows and Linux. Key Solutions: ·         Application Level: High availability with restart scripts per application. ·         Hypervisor Level: High availability without restart scripts per application. ·         Container or Pod Level: High availability without restart scripts per application. SafeKit is ideal for software publishers, resellers, and distributors looking to enhance their products with high availability features. It also offers an OEM opportunity for partners to integrate SafeKit into their own applications. ### 1.1.2         SafeKit solutions  See here for a list of SafeKit solutions. | | | | --- | --- | | **Application-level HA** In this type of solution, only application data is replicated. And only the application is restarted in case of a failure. Integration tasks must be implemented, including defining the list of services to restart in case of failover, specifying data folders for replication, configuring software checkers, and setting up a virtual IP address. This solution is platform-independent and works with applications inside physical machines, virtual machines, in the cloud. Any hypervisor is supported (e.g., VMware, Hyper-V, etc.). | **Virtual machine-level HA** In this type of solution, the entire virtual machine (VM) is replicated, including the application and OS. The complete virtual machine is restarted in case of a failure. The advantage is that there is no need to have in-depth knowledge of the application (services to restart, location of application data to replicate), and no virtual IP address needs to be configured. This solution works with Windows/Hyper-V and Linux/KVM but not with VMware. This is an active/active solution with multiple virtual machines replicated and restarted between the two nodes. | Note: Applications running in containers or pods also do not require dedicated restart scripts. SafeKit provides generic restarts and real-time replication of persistent data for these environments (see the list of SafeKit solutions). ### 1.1.3         SafeKit architectures SafeKit offers two basic high availability clusters for Windows and Linux: ·         the mirror cluster, with real-time file replication and failover, built by deploying a mirror module on 2 servers, ·         the farm cluster, with network load balancing and failover, built by deploying a farm module on 2 servers or more. Several modules can be deployed on the same cluster. Thus, advanced clustering architectures can be implemented: ·         the farm+mirror cluster built by deploying a farm module and a mirror module on the same cluster, ·         the active/active cluster built by deploying several mirror modules on 2 servers, ·         the N-1 cluster built by deploying N mirror module on N+1 servers. Specific clusters are also interesting to consider with SafeKit: ·         the Hyper-V or KVM cluster with real-time replication and failover of entire virtual machines between 2 active hypervisors, ·         mirror or farm clusters in the Cloud. ### 1.1.4         SafeKit cluster definition A SafeKit cluster is a set of servers where SafeKit is installed and running. All servers within a given SafeKit cluster share the same cluster configuration, which includes the list of servers and networks used. These servers communicate with each other to maintain a global view of the configurations of the SafeKit modules. A server cannot belong to multiple SafeKit clusters simultaneously. Configuring the cluster is a prerequisite before the installation and configuration of SafeKit modules. This can be done using the SafeKit web console or through online commands. ### 1.1.5         SafeKit application module definition An application module is a customization of SafeKit for a specific application or hypervisor. See here for a list of modules and their quick installation guides. **Types of Modules** ·         Generic farm and mirror modules for new applications, ·         Preconfigured application modules for databases, web servers…, ·         Hypervisors modules (hyperv.safe, kvm.safe) for real-time replication and restart of entire virtual machines. **Module Contents** In practice, an application module is a “.safe” file (zip type) that includes: ·         The configuration file userconfig.xml, which contains: o    The virtual IP address (not necessary for a hypervisor module), o    File directories to replicate in real time (for a mirror module), o    Network load balancing criteria (for a farm module), o    Configuration of software and hardware failures detectors, ·         The scripts to start and stop an application or a virtual machine. **Deployment Steps** Once the application module is configured and tested, deployment requires no specific IT skills: ·         Install the application or the hypervisor on 2 standard servers, ·         Install the SafeKit software on both servers, ·         Install the module on both servers. Configuring, deploying, and monitoring modules can be done using the SafeKit web console or through online commands. ### 1.1.6         SafeKit limitations | | | | | | --- | --- | --- | --- | | **Typical usage with SafeKit** | | | | | Replication of a few Tera-bytes | Replication < 1 million files | Replication <= 32 virtual machines | 1 or 10 G/s LAN or extended LAN | | **Limitation** | | | | | Resynchronization after a failure takes too long. On a 1 Gb/s network, 3 Hours for 1 Tera-bytes. On a 10 Gb/s network, 1 hour or less for 1 Tera-bytes (depends on write disk IO performances). | Resynchronization after a failure takes too long. Time to check each file between both nodes. | In full virtual machine replication mode, and with one virtual machine in a mirror module, the limit is 32 modules per cluster. | Failover of the virtual IP address is built-in when in the same subnet. A LAN provides adequate bandwidth for resynchronization. A LAN provides adequate latency (typically a round-trip of less than 2ms) for synchronous replication. | | **Alternative** | | | | | Use shared storage. | Put files in a virtual hard disk replicated by SafeKit. | Use another HA solution with shared storage. | Use backup solutions with asynchronous replication. | ## 1.2             The SafeKit mirror cluster ### 1.2.1         Real time file replication and application failover The mirror cluster is an active-passive high-availability solution, built by deploying a mirror module within a two-node cluster. The application runs on a primary server and is restarted automatically on a secondary server if the primary server fails. With its real-time file replication function, this architecture is particularly suited to providing high availability for back-end applications with critical data to protect against failure. Microsoft SQL Server, PostgreSQL, MariaDB, Oracle, Milestone, Nedap, Docker, Podman, Hyper-V, and KVM solutions are examples of mirror modules. You can create your own mirror module for your application based on the generic mirror.safe module. See here for a list of modules. Note that Hyper-V and KVM mirror modules replicate entire virtual machines, including applications and operating systems. They do not require a virtual IP, as the VM restart handles the failover of the VM physical IP address. The mirror cluster works as follows. ### 1.2.2         Step 1. Normal operation ![](safekituserguideen_fichiers/image012.jpg) Server 1 (PRIM) runs the application. SafeKit replicates files opened by the application. Only changes made by the application in the files are replicated in real time across the network, thus limiting traffic. For replication, only names of file directories to replicate are configured in SafeKit. There are no pre-requisites on disk organization for the two servers. Directories to replicate may be located in the system disk. ### 1.2.3         Step 2. Failover **![](safekituserguideen_fichiers/image013.jpg)** When Server 1 fails, Server 2 takes over. SafeKit switches the virtual IP address and restarts the application automatically on Server 2. The application finds the files replicated by SafeKit up-to-date on Server 2, thanks to the synchronous replication between Server 1 and Server 2. The application continues to run on Server 2 by locally modifying its files that are no longer replicated to Server 1. The switch-over time is equal to the fault-detection time (set to 30 seconds by default) plus the application start-up time. Unlike disk replication solutions, there is no delay for remounting file systems and running recovery procedures. ### 1.2.4         Step 3. Failback and automatic resynchronization **![](safekituserguideen_fichiers/image014.jpg)** Failback involves restarting Server 1 after fixing the problem that caused it to fail. SafeKit automatically resynchronizes the files, updating only the files modified on Server 2 while Server 1 was halted. This automatic reintegration takes place without stopping the application, which can continue running on Server 2. This is a major feature that differentiates SafeKit from other solutions, which require manual operations to reintegrate Server 1 in the cluster. ### 1.2.5         Step 4. Return to normal operation ![](safekituserguideen_fichiers/image015.png) After reintegration, the files are once again in mirror mode, as in step 1. The system is back in high-availability mode, with the application running on Server 2 and SafeKit replicating file updates to Server 1. If administrators want the application to run on Server 1, they can execute a ‘Stop/Start’ command on the PRIM server either through the console at the appropriate time or automatically by configuring a default primary server. ### 1.2.6         Synchronous replication versus asynchronous replication There is a significant difference between synchronous replication, as offered by the SafeKit mirror solution, and asynchronous replication traditionally offered by other file replication solutions. With synchronous replication, when a disk IO is performed by the application on the primary server inside a replicated file, SafeKit waits for the IO acknowledgement from the local disk and from the secondary server, before sending the IO acknowledgement to the application. This mechanism is essential for recovery of transactional applications. The latency of a LAN (typically a round-trip of less than 2ms) between the servers is required to implement synchronous data replication, possibly with an extended LAN in two geographically remote computer rooms. With asynchronous replication implemented by other solutions, the IOs are placed in a log on the primary server but the primary server does not wait for the IO acknowledgments of the secondary server. Thus, all data that has not been copied over the network to the second server is lost in the event of a failure of the first server. In particular, a transactional application may lose committed data in the event of a failure. Asynchronous replication can be used for data replication over a low-speed WAN to back up data remotely, but it is not suitable for high availability with automatic failover. SafeKit provides a semi-synchronous solution, implementing the asynchrony not on the primary server but on the secondary one. In this solution, SafeKit always waits for the acknowledgement of the two servers before sending the acknowledgement to the application. But on the secondary, there are 2 options asynchronous or synchronous. In the asynchronous case, the secondary sends the acknowledgement to the primary upon receipt of the IO and writes to disk after. In the synchronous case, the secondary writes the IO to disk and then sends the acknowledgement to the primary. The synchronous mode is required if we consider a simultaneous double power outage of two servers, with inability to restart the former primary server and requirement to re-start on the secondary. ### 1.2.7         Behavior in case of network isolation A **heartbeat** is a mechanism for synchronizing two servers and detecting failures by exchanging data over a shared network. If one server loses all heartbeats, it assumes the other is down and runs the application ALONE. SafeKit supports multiple heartbeats across shared networks. A dedicated network with a second heartbeat can prevent network isolation and be used as the replication network. **Network Isolation**: ·         Upon losing all heartbeats, both servers transition to the ALONE state, running the application independently. ·         After the isolation, one server stops and resynchronizes data from the other server. ·         The cluster returns to PRIM-SECOND state. **Splitbrain Checker**: ·         Uses a witness IP (usually a router) to avoid double execution during isolation. ·         Only the server with witness access goes ALONE, the other waits. ·         After isolation, the WAIT server resynchronizes and becomes SECOND. ### 1.2.8         3-node replication SafeKit only supports replication between two nodes. However, it is possible to implement 3-node replication by combining SafeKit with a backup solution. An application is made highly available between 2 nodes thanks to SafeKit with its synchronous real-time replication (no data loss) and automatic failover. Additionally, a backup solution is implemented for asynchronous replication to a third node in a disaster recovery site. Since there is data loss with an asynchronous backup solution, the failover to the third node is manual and decided by an administrator. Note that the real-time replication of SafeKit does not eliminate the need for a backup solution. For example, a ransomware attack encrypting replicated data on the primary server will also encrypt data on the secondary server in real-time with SafeKit. Only a backup solution with a retention policy can resolve a ransomware attack. The administrator must restore the backup from before the ransomware attack. ### 1.2.9         SafeKit on a single node to protect against software failures You can configure an application module in "light" mode, which corresponds to a module running on a single node without synchronizing with other nodes (unlike mirror or farm modules). A light module includes the start and stop of an application, as well as SafeKit checkers that detect software errors and perform automatic restarts on a single node. The light module interfaces with the SafeKit console, allowing an administrator to view the status of the application module and manually trigger application restarts using a button-click interface. There is no need to define a virtual IP address or replicated directories in a light module. Note that this can also serve as a first step before transitioning to a mirror module or a farm module. ## 1.3             The SafeKit farm cluster ### 1.3.1         Network load balancing and application failover ![](safekituserguideen_fichiers/image016.jpg) The farm cluster is an active-active high-availability solution, built by deploying a farm module within a cluster of two or more nodes. The farm cluster provides both network load balancing, through transparent distribution of network traffic, and software and hardware failover. This architecture offers a simple solution to support the increase in system load. The same application runs on each server, and the load is balanced by the distribution of network activity on the different servers of the farm. Farm clusters are suited to front-end applications like web services. Apache, Microsoft IIS, NGINX solutions are examples of farm modules. You can write your own farm module for your application, based on the generic farm.safe module. See here for a list of modules. ### 1.3.2         Principle of a virtual IP address with network load balancing The virtual IP address is configured locally on each server in the farm. Input traffic for this address is distributed among all servers by a filter within each server’s kernel. The load balancing algorithm inside the filter is based on the identity of the client packets (client IP address, client TCP port). Depending on the identity of the client packet, only one filter on a server accepts the packet. Once a packet is accepted by the filter on a server, only the CPU and memory of that server are used by the application responding to the client’s request. The output messages are sent directly from the application server to the client. If a server fails, the SafeKit heartbeat protocol in a farm reconfigures the filters to re-balance the traffic among the remaining available servers. ### 1.3.3         Load balancing for stateful or stateless web services With a stateful server, session affinity is required. The same client must connect to the same server across multiple TCP sessions to retrieve its context. In this scenario, the SafeKit load balancing rule is configured on the client IP address. This ensures that the same client always connects to the same server for multiple TCP sessions, while different clients are distributed across various servers in the farm. This configuration is used when session affinity is required. With a stateless server, there is no session affinity. The same client can connect to different servers in the farm across multiple TCP sessions, as no context is stored locally on a server from one session to another. In this case, the SafeKit load balancing rule is configured on the TCP client session identity. This configuration is optimal for distributing sessions between servers but requires a TCP service without session affinity. ### 1.3.4         Chain high availability solution in a farm What is a chain HA solution (also known as a cascading HA solution)? ·         Multiple servers are linked in a sequence: If one server fails, the next one in the chain takes over. ·         Priority-based management: A single server, the one with the highest priority in the chain and which is available, manages all requests from clients. ·         Failover process: If the server with the highest priority fails, the next available server with the highest priority takes over. ·         Reintegration: When a server comes back online and has the highest priority, it resumes handling all client requests. ·         Quick recovery time: This solution has a quick recovery time, as the application is pre-started on all servers. The recovery time is essentially the time needed to reconfigure the priorities among the servers in the farm (a few seconds). ·         Replication limitations: This solution does not support real-time replication, which is limited to mirror architecture. However, a combined farm+mirror architecture is available. To implement a chain high availability solution, SafeKit offers a "power" variable in the load balancing rules, which is set at the level of each server in the cluster. The power variable allows you to allocate more or less traffic to a server. When the power variable is set as a multiple of 64 between servers (e.g., 1, 64, 64\*64, 64\*64\*64, ...), the chain high availability solution is implemented. ## 1.4             Clusters running several modules ### 1.4.1         The SafeKit farm+mirror cluster **Network load balancing, file replication and application failover** You can mix farm and mirror modules on the same cluster. This option allows you to implement a multi-tier application architecture, such as apache\_farm.safe (farm architecture with load balancing and failover) and postgresql.safe (mirror architecture with file replication and failover) on the same servers. ![](safekituserguideen_fichiers/image017.jpg) As a result, load balancing, file replication and failover are managed coherently on the same servers. ### 1.4.2         The SafeKit active/active cluster with replication **Crossed replication and mutual failover** In an active / active cluster with replication, there are two servers and two mirror modules in mutual failover (appli1.safe and appli2.safe). Each application server is backup of the other server. ![](safekituserguideen_fichiers/image018.jpg) If one application server fails, both applications will run on the same physical server. Once the failed server is restarted, its application will return to its default primary server. A mutual failover cluster is more cost-effective than two separate mirror clusters, as it eliminates the need for backup servers that remain idle most of the time, waiting for a primary server to fail. However, in the event of a server failure, the remaining server must be capable of handling the combined workload of both applications. Note that: ·         Both applications, Appli1 and Appli2, must be installed on each server to enable application failover. ·         This architecture is not limited to just two applications; N application modules can be deployed across two servers. ·         Each mirror module will have its own virtual IP address, its own replicated file directories, and its own restart scripts. ### 1.4.3         The SafeKit N-1 cluster **Replication and application failover from N servers to 1** In an N-1 cluster, N mirror application modules are deployed across N primary servers and a single backup server. ![](safekituserguideen_fichiers/image019.jpg) In the event of a failure, unlike in an active/active cluster, the backup server does not need to manage a double workload when a primary server fails. This assumes only one failure occurs at a time. While the solution can support multiple primary server failures simultaneously, in such cases, the single backup server will need to handle the combined workload of all the failed servers. In a N-1 cluster, there are N mirror application modules installed between N primary servers and one backup server. Note that: ·         All applications (Appli1, Appli2, Appli3) must be installed on the single backup server to enable application failover. ·         Each mirror module will have its own virtual IP address, its own replicated file directories, and its own restart scripts. ## 1.5             The SafeKit Hyper-V or KVM cluster ### 1.5.1         Load balancing, replication, failover of entire virtual machines The Hyper-V or KVM cluster is an example of an active-active cluster. Multiple applications can be hosted in various virtual machines, which are replicated and restarted by SafeKit. Each virtual machine is managed by SafeKit within its own mirror module. ![](safekituserguideen_fichiers/image020.png) The solution has the following features: ·         Real-time synchronous replication of entire virtual machines with failover capabilities. ·         A centralized, user-friendly SafeKit console for managing all VMs, including the ability to migrate VMs between servers to optimize load distribution. ·         A checker for each VM to detect if it has locked up, crashed, or ceased to function, and to restart the VM if necessary. ·         An attractive solution that requires no application integration. ·         A robust architecture suitable for high-availability solutions that cannot be integrated at the application level. A free trial of the Hyper-V cluster with SafeKit is available here. A free trial of the KVM cluster with SafeKit is available here. ## 1.6             SafeKit clusters in the cloud For a full description, refer to section 16. ### 1.6.1         Mirror cluster in Azure, AWS and GCP SafeKit delivers high-availability clusters with real-time replication and failover in Azure, AWS, and GCP through the deployment of a mirror module. ![](safekituserguideen_fichiers/image021.png) The mirror solution in the cloud is similar to the on-premise one, except that the virtual IP address must be configured at the load balancer level: ·         Virtual machines are placed in different availability zones, which are in different subnets. ·         The critical application runs on the primary server. ·         Users connect to a primary/secondary virtual IP address managed by the cloud load balancer. ·         SafeKit provides a health check configured in the load balancer. On the primary server, the health check returns OK to the load balancer, while it returns nothing on the secondary server. Thus, all requests to the virtual IP address are routed to the primary server. ·         If the primary server fails or is stopped, the secondary server automatically becomes the primary one and returns OK to the health check. Thus, all requests to the virtual IP address are rerouted to the new primary server. ·         SafeKit monitors the critical application on the primary server using SafeKit checkers. ·         SafeKit automatically restarts the critical application in the event of software or hardware failure, thanks to restart scripts. ·         SafeKit performs synchronous real-time replication of files containing critical data. For more information, refer to mirror cluster in Azure, mirror cluster in AWS or mirror cluster in GCP. ### 1.6.2         Farm cluster in Azure, AWS and GCP SafeKit delivers high-availability clusters with network load balancing and failover in Azure, AWS, and GCP through the deployment of a farm module. ![](safekituserguideen_fichiers/image022.png) The farm solution in the cloud is similar to the on-premise one, except that the virtual IP address must be configured at the load balancer level: ·         Virtual machines are placed in different availability zones, which are in different subnets. ·         The critical application runs on all servers. ·         Users are connected to a virtual IP address managed by the cloud load balancer. ·         SafeKit provides a health check configured in the load balancer. The health check returns OK on all servers running the application. ·         If a server fails or is stopped, the checker returns nothing to the load balancer, which then stops routing requests to that server. ·         SafeKit monitors the critical application on all servers using SafeKit checkers. ·         SafeKit automatically restarts the critical application on a server when there is a software failure, thanks to restart scripts. For more information, refer to farm cluster in Azure, farm cluster in AWS or farm cluster in GCP. # 2. Installation ![*](safekituserguideen_fichiers/image001.png)       Section 2.1 “SafeKit install” ![*](safekituserguideen_fichiers/image001.png)       Section 2.2 “Mirror installation recommendation” ![*](safekituserguideen_fichiers/image001.png)       Section 2.3 “Farm installation recommendation” ![*](safekituserguideen_fichiers/image001.png)       Section 2.4 “SafeKit upgrade” ![*](safekituserguideen_fichiers/image001.png)       Section 2.5 “SafeKit full uninstall” ![*](safekituserguideen_fichiers/image001.png)       Section 2.6 “SafeKit documentation” ## 2.1             SafeKit install ### 2.1.1         Download the package 1.    Connect to SafeKit product download area 2.    Go to /Platforms//Current versions 3.    Download the package | | | | --- | --- | | Windows | Two packages are available: ·         safekit\_windows\_x86\_64\_8\_2\_x\_y.msi A Windows Installer package. This package requires the Visual Studio 2022 C Runtime to be pre-installed on the system. ·         safekit\_windows\_x86\_64\_8\_2\_x\_y.exe A standalone executable bundle. It includes both the SafeKit package and the required Visual Studio 2022 C Runtime, making it suitable for systems without prior runtime installation. Choose one or the other package depending on whether the VS2022 C runtime is installed or not. | | Linux | Two files are provided: ·         safekitlinux\_x86\_64\_8\_2\_x\_y.bin A self-extractable file containing the Linux packages and associated installation script. ·         safekitlinux\_x86\_64\_8\_2\_x\_y.sha256 A file containing the SHA-256 checksum used to verify the integrity of the .bin file. | | | | --- | --- | | Commentaire, ajouter contour | You can verify the integrity of the file with the command: sha256sum -c safekitlinux\_x86\_64\_8\_2\_x\_y.sha256 | ·         Key Id (e.g. 779244d710b22ada)  The long-format identifier of the GPG public key used to sign SafeKit RPM packages starting from version 8.2.5 for Red Hat. This signature ensures that users install a verified, untampered package from Eviden. | | | | --- | --- | | Commentaire, ajouter contour | You can verify that the RPM is properly signed with SafeKit key using the command below. It displays the Key ID of the signing key. rpm -qi SafeKit-8.2.5-x.el9.x86\_64.rpm … Signature: RSA/SHA256, …, Key ID 779244d710b22ada … | | ### 2.1.2         Installation directories and disk space provisioning SafeKit is installed in: | | | | | --- | --- | --- | | SAFE | ·         in Windows SAFE=C:\safekit if %SYSTEMDRIVE%=C: ·         in Linux SAFE=/opt/safekit | Minimum free disk space: 97MB | | SAFEVAR | ·         in Windows SAFEVAR= C:\safekit\var if %SYSTEMDRIVE%=C: ·         in Linux SAFEVAR=/var/safekit | Minimum free disk space: 20MB + at least 20MB (up to 3 GB) per module for dumps |   ### 2.1.3         SafeKit install procedure #### 2.1.3.1      Install on Windows as administrator The SafeKit installation logs can be found in SAFEVAR directory. ##### 2.1.3.1.1                SafeKit package install 1.    Log-in as administrator on Windows server 2.    Locate the downloaded file safekit\_windows\_x86\_64\_8\_2\_x\_y.msi (or safekit\_windows\_x86\_64\_8\_2\_x\_y.exe) 3.    Install in interactive mode by double-clicking it and go through the installer wizard Before SafeKit 8.2.3, after installation, you need to run the firewall configuration scripts (see section 10.3) and initialize the SafeKit web service (see section 11.2.1.2). Since SafeKit 8.2.3, at the end of the SafeKit Setup, you will be asked to check or uncheck " Set console credentials and firewall rules now ". ![](safekituserguideen_fichiers/image025.png) If the box is checked, when clicking the “Finish” button: o    it configures Microsoft Windows Firewall for SafeKit. For details or other firewalls, see section 10.3. o    it opens a window to enter the password for the admin user of the SafeKit web console. ![](safekituserguideen_fichiers/image026.jpg) This step is mandatory to initialize the default configuration of the web service that requires authentication. It is initialized with the admin user and the given password pwd, for instance. It then allows access to all the web console's features, by logging in with admin/pwd, and run distributed commands. For details, see section 11.2.1. | | | | --- | --- | | Commentaire important contour | The password must be identical on all nodes that belong to the same SafeKit cluster. Otherwise, web console and distributed commands will fail with authentication errors. | **or** 3.    Install in non-interactive mode, by executing: msiexec /qn /i safekitwindows\_8\_2\_x\_y.msi Then, the firewall setup and web service initialization must be done. ##### 2.1.3.1.2                Firewall setup This step is mandatory to enable communication between the nodes of the SafeKit cluster and with the web console. No action required when firewall automatic configuration has been performed during the package install. Otherwise see section 10.3. ##### 2.1.3.1.3                Web service initialization This step is mandatory to initialize the default configuration of the web service, which is accessed by the web console and the global safekit command. The web service requires authentication to access the service. No action required when the web service initialization has been performed during the package installation. Otherwise, see section 11.2.1.2. ##### 2.1.3.1.4                Antivirus setup This step is necessary if the server's antivirus interferes with the operation of SafeKit. Since this issue (described in section 7.22) may occur sporadically but not systematically with Windows Defender, it is recommended to configure it to prevent any malfunction. Refer to section 10.6 for the list of legitimate SafeKit directories and processes that should not be affected by the antivirus. #### 2.1.3.2      Install on Linux as root Since SafeKit 8.2.6, the SafeKit installation logs are generated in /tmp/safekit directory. ##### 2.1.3.2.1                SafeKit package install 1.    Open a Shell console as root on Linux server 2.    Go to the directory that contains the downloaded file safekitlinux\_x86\_64\_8\_2\_x\_y.bin auto extractible zip file 3.    Run chmod +x safekitlinux\_x86\_64\_8\_2\_x\_y.bin 4.    Run./safekitlinux\_8\_2\_x86\_64\_x\_y.bin it extracts the package and the safekitinstall script 5.    Install in interactive mode by executing ./safekitinstall During the installation: ·         Since SafeKit 8.2.5 on Red Hat, reply to “Do you accept to import this public key (yes|no)?” SafeKit RPM packages are now GPG-signed. By replying yes, the script will automatically import the SafeKit GPG public key and will continue the installation process. If you reply no, it will be interrupted. | | | | --- | --- | | Commentaire, ajouter contour | The script displays the identifier of the key (Key Id) used to sign the RPM package. You can check this value against the official one published in the SafeKit product download area. |   | | | | --- | --- | | Commentaire, ajouter contour | The public GPG key used to sign the RPM package is extracted at the same time as the package into the file named RPM-GPG-KEY-SafeKit. You can retrieve its Key Id with the command: gpg --show-keys --no-options RPM-GPG-KEY-SafeKit (The Key Id corresponds to the last 16 characters of the pub line) | ·         reply to “Do you accept that SafeKit automatically configure the local firewall to open these ports (yes|no)?” If you answer yes, it configures firewalld or iptable Linux firewall for SafeKit. For details or other firewalls, see section 10.3. ·         reply to “Please enter a password or "no" if you want to set it later” This step is mandatory to initialize the default configuration of the web service. The web service requires authentication to access the service. It initializes it with the admin user and the given password pwd, for instance. It then allows access to all the web console's features, by logging in with admin/pwd, and run distributed commands. For details, see section 11.2.1. | | | | --- | --- | | Commentaire important contour | The password must be identical on all nodes that belong to the same SafeKit cluster. Otherwise, web console and distributed commands will fail with authentication errors. | **or** 5.    Install in non-interactive mode, by executing: ./safekitinstall -q Use the option -passwd pwd for initializing the web service authentication (where pwd is the password set for the admin user). Then, the firewall setup must be manually done. | | | | --- | --- | | Commentaire important contour | Starting with SafeKit version 8.2.5, on Red Hat, RPM packages are GPG-signed. Thus, the SafeKit GPG public key is automatically imported to allow the installation to continue. | ##### 2.1.3.2.2                Firewall setup This step is mandatory to enable communication between the nodes of the SafeKit cluster and with the web console. No action is required when firewall automatic configuration has been performed during the package installation. Otherwise see section 10.3. ##### 2.1.3.2.3                Web service initialization This step is mandatory to initialize the default configuration of the web service, which is accessed by the web console and the global safekit command. The web service requires authentication to access the service. No action required when the web service initialization has been performed during the package installation. Otherwise, see section 11.2.1.2. ##### 2.1.3.2.4                Antivirus setup This step is necessary if the server's antivirus interferes with the operation of SafeKit (see section 7.22). Refer to section 10.6 for the list of legitimate SafeKit directories and processes that should not be affected by the antivirus. ### 2.1.4         Use the SafeKit web console or command line interface Once installed, the SafeKit cluster must be defined. Then modules can be installed, configured, and administered. All these actions can be done with the SafeKit console or the command line interface. #### 2.1.4.1      The SafeKit web console 1.    Start a web browser (Microsoft Edge, Firefox, or Chrome) 2.    Connect it to the URL http://host:9010 (where host is the name or IP address of one of the SafeKit nodes) 3.    In the login page, enter admin as user’s name and the password you gave on initialization (e.g., pwd) 4.    Once the console is loaded, the admin user can access to ![](safekituserguideen_fichiers/image032.png)Monitoring and ![](safekituserguideen_fichiers/image033.png)Configuration in the navigation sidebar, as he has the default Admin role For details see section 3. #### 2.1.4.2      The SafeKit command line interface It is based on the single safekit command located at the root of the SafeKit installation directory. Almost all safekit commands can be applied locally or on a list of nodes in the SafeKit cluster. This is called global or distributed command. For details on the safekit command, see section 9.   To use the safekit command: | | | | --- | --- | | In Windows | 1.    Open a PowerShell console as administrator 2.    Go to the root of the SafeKit installation directory SAFE (by default SAFE=C:\safekit if %SYSTEMDRIVE%=C:) cd c:\safekit 3.    Run .\safekit.exe for the local command 4.    Run .\safekit.exe -H "" for the command distributed across multiple nodes | | In Linux | 1.    Open a Shell console as root 2.    Go to the root of the SafeKit installation directory SAFE (by default SAFE=/opt/safekit) cd /opt/safekit 3.    Run ./safekit for the local command 4.    Run ./safekit -H "" for the command distributed across multiple nodes |   For instance, to display the levels (SafeKit, OS…): ·         for the local host safekit level ·         for all hosts configured in the SafeKit cluster safekit -H "\*" level ### 2.1.5         SafeKit license keys License keys are determined and verified based on the Operating System (Windows or Linux) and the hostnames of machines (not the FQDN), as returned by the `hostname` command in a Windows command prompt or a Linux shell. They are delivered in a text file. Once the license key file is installed, there is no need for a connection to a license server. ·         If you do not install any license key file, the product will stop functioning every 3 days. However, it can be restarted for another 3 days. ·         You can download a one-month trial key file here. ·         When a license key expires or is incorrect (e.g., wrong OS or hostname), the system falls into the 3-day behavior. ·         After placing a purchase order, you obtain a permanent key file. The permanent key file can be installed without reinstalling or stopping the product. ·         The key file can contain keys for multiple hostnames. SafeKit will detect the appropriate license for the correct OS/hostname on each server. ·         Save the key file into the `SAFE/conf/license.txt` file (or any other file in `SAFE/conf`) on each server. ·         If files in `SAFE/conf` contain more than one key file, the most favorable key will be chosen. ·         Check the key conformance on each server with the command `SAFE/safekit level` or with the SafeKit web console. ### 2.1.6         System specific procedures and characteristics #### 2.1.6.1      Windows ·         Apply a special procedure to properly stop SafeKit modules at machine shutdown and to start safeadmin service at boot: see section 10.4. ·         For network interfaces with teaming and with SafeKit load balancing, it is necessary to uncheck "Vip" on physical network interfaces of teaming and keep it checked only on teaming virtual interface. #### 2.1.6.2      Linux ·         In Linux, the SafeKit package depends on other system packages. Most of them are installed automatically, except those specific to the implementation of load balancing in a farm and file replication in a mirror. ·         For an updated list of required packages, see the *SafeKit Release Notes**.* ·         The user safekit and a group safekit are created: all users belonging to the safekit group, and the user root can execute SafeKit commands ·         In a farm module with load balancing on a virtual IP address, the vip kernel module is compiled when the module is configured. To compile successfully, Linux packages must be installed. See the *SafeKit Release Notes*for an up-to-date list of the packages. ·         For a farm with SafeKit load balancing on a bonding interface, no ARP should be set in the bonding configuration. Otherwise, the association is broken in client ARP caches with physical MAC address of the bonding interface. ·         When Secure Boot is enabled and you are using a farm module, follow the procedure described in section 10.5 to sign and enroll the SafeKit kernel modules that implement load-balancing. ## 2.2             Mirror installation recommendation | | | | --- | --- | | ip 1.1     ip 1.2 | virtual ip =      ip 1.10                           mirror(app1)=  app1                                                              dir1                dir1 | ### 2.2.1         Hardware and system prerequisites ·         2 servers with the same operating System ·         Supported OS ·         Disk drive with write-back cache recommended for the performance of the IOs ### 2.2.2         Network prerequisites ·         1 physical IP address per server (ip 1.1 and ip 1.2) ·         If you need to set a virtual IP address (ip 1.10), both servers must be in the same IP network with the standard SafeKit configuration (LAN or extended LAN between two remote computer rooms). For setting a virtual IP address with servers in different IP networks, see section 13.6.3. ### 2.2.3         Application prerequisites ·         The application is installed and starts on both servers ·         Application can be started and stopped using command lines ·         On Linux, command lines like service "service" start|stop or su -user "appli-cmd" ·         On Windows, command lines like net start|stop "service" ·         If necessary, application with a procedure to recover after crash ·         Remove automatic application start at boot and configure the boot start of the module instead ### 2.2.4         File replication prerequisites ·         File directories that will be replicated are created on both servers ·         They are located at the same place on both servers in the file tree ·         It is better to synchronize clocks of both server for file replication (NTP protocol) ·         On Linux, align uids/gids on both servers for owners of replicated directories/files ·         See also system specific procedures and characteristics in section 2.1.6 ## 2.3             Farm installation recommendation | | | | --- | --- | | ip 1.1    ip 1.2   ip 1.3 | virtual IP =             ip 1.20     ip 1.20      ip 1.20                                                                farm (app2) =          app2        app2         app2 | ### 2.3.1         Hardware and system prerequisites ·         At least 2 servers with the same operating System ·         Supported OS ·         Linux: kernel compilation tools installed for vip kernel module ### 2.3.2         Network prerequisites ·         1 physical IP address per server (ip 1.1, ip 1.2, ip 1.3) ·         If you need to set a virtual IP address (ip 1.20), servers must be in the same IP network with the standard SafeKit configuration (same LAN or extended LAN between remote computer rooms). For setting a virtual IP address with servers in different IP networks, see section 13.6.3. ·         See also system specific procedures and characteristics in section 2.1.6 ### 2.3.3         Application prerequisites The same prerequisites as for a mirror module described in section 2.2.3 ## 2.4             SafeKit upgrade If you encounter a problem with SafeKit, see the *Software Release Bulletin*containing the list of fixes on the product. If you want to take advantage of some new features, see the *SafeKit Release Notes*. This document also tells you if you are in the case of a major upgrade (ex. 7.5 to 8.2) which requires a different procedure from the one presented here. The upgrade procedure consists in uninstalling the old package and then installing the new package. All nodes in the same cluster must be upgraded. ### 2.4.1         Prepare the upgrade 1.    Note the state "on" or "off" of SafeKit services and modules started automatically at boot safekit boot webstatus; safekit boot status -m *AM* (where *AM* is the name of the module) and in Windows: safekit boot snmpstatus; | | | | --- | --- | | Commentaire important contour | The start at boot of the module can be defined in its configuration file. If so, the use of the safekit boot command becomes unnecessary. | 2.    for a mirror module note the server in the ALONE or PRIM status to know which server holds the up-to-date replicated files 3.    optionally, take snapshots of modules Uninstalling/reinstalling will reset logs and dumps of each module. If you want to keep this information (logs and last 3 dumps and configurations), run the command safekit snapshot -m *AM* /path/snapshot\_xx.zip (replace *AM* by the module name) ### 2.4.2         Uninstall procedure On Windows as administrator and on Linux as root: 1.    stop all modules using the command safekit shutdown For a mirror in the PRIM-SECOND status, stop first the SECOND server to avoid an unnecessary failover 2.    close all editors, file explorers, shells, or terminal under SAFE and SAFEVAR (to avoid package uninstallation error) 3.    uninstall SafeKit package | | | | --- | --- | | Windows | Use the Control Panel-Add/Remove Programs applet | | Linux | Use the command safekit uninstall |   4.    undo all configurations that you have done manually for the firewall setup (see section 10.3) Uninstalling SafeKit includes creating a backup of the installed modules in SAFE/Application\_Modules/backup, then unconfiguring them. ### 2.4.3         Reinstall and postinstall procedure 1.    Install the new package as described in section 2.1 2.    Check with the command safekit level the installed SafeKit version and the validity of the license (which has not been uninstalled) If you have a problem with the new package and the old key, take a temporary license: see section 2.1.5 3.    If you use the web console, clear the browser cache and refresh pages in the web browser 4.    Since SafeKit 8.2.1, previously configured modules are automatically reconfigured on upgrade. However, you may still need to reconfigure module to apply any configuration changes coming with the new version (see the *SafeKit Release Notes*). Reconfigure the module either with: o    the web console by navigating to ![](safekituserguideen_fichiers/image033.png)“Configuration/Modules configuration/ ![](safekituserguideen_fichiers/image040.png)Configure the module/” o    the web console by directly entering the URL http://host:9010/console/en/configuration/modules/*AM*/config/ o    the command safekit config -m *AM* where *AM* is the module name 5.    If necessary, reconfigure the automatic start of modules at boot The start at boot of the module can be defined in its configuration file. If so, skip this step. Otherwise, run the command safekit boot -m *AM* on (replace *AM* by the module name) 6.    Restart the modules | | | | --- | --- | | Mirror module | The module must be started as primary on the node with the updated replicated files (former PRIM or ALONE) either with: ·         the web console by navigating to Monitoring/of the node/Force start/As primary ·         the command safekit prim -m *AM* (replace *AM* by the module name)   Check that the application is working properly once the module is in ALONE state, before starting the other node. On the other node (former SECOND), the module must be started in secondary mode either with: ·         the web console by navigating to Monitoring/of the node/Force start/As secondary ·         the command safekit second -m *AM* (replace *AM* by the module name) Once this initial start has been performed by selecting the primary and secondary nodes, subsequent starts can be performed with: ·         the web console by navigating to Monitoring/of the node/Start/ ·         the command safekit start -m *AM* (replace *AM* by the module name) | | Farm module | Start the module either with: ·         the web console by navigating to Monitoring/of the module/Start/ ·         the command safekit start -m *AM* (replace *AM* by the module name) |   Furthermore, in cases where you have modified the setup of the SafeKit web service, SNMP monitoring or SafeKit notification agent: 1.    the SafeKit web service safewebserver ·         If its automatic start at boot had been disabled, disable it again with the command safekit boot weboff ·         If you had modified configuration files and these have evolved in the new version, your modifications are saved into SAFE/web/conf before being overwritten by the new version. Carrying over your old configuration to the new version may require some adaptations. For details on the default setup and all predefined setups, see section 11. ·         For HTTPS and login/password configurations, certificates, and user.conf / group.conf generated for the previous release should be compatible. 2.    The SafeKit SNMP monitoring ·         In Windows, if its automatic start at boot had been enabled, enable it again with the command safekit boot snmpon ·         If you had modified configuration files and these have evolved in the new version, your modifications are saved into SAFE/snmp/conf before being overwritten by the new version. Carrying over your old configuration to the new version may require some adaptations. For details, see section 10.11. 3.    The SafeKit email notification agent Since SafeKit 8.2.4, SafeKit offers a notification agent to send emails for major modules events. For details, see section 10.10. If you have enabled it, it will remain enabled after the upgrade. However, you may need to reconfigure the SafeKit notification agent, as described in section 10.10.1, if its configuration file has evolved between the two versions. ## 2.5             SafeKit full uninstall For completely removing the SafeKit package, follow the procedure described below. ### 2.5.1         Uninstall on Windows as administrator 1.    Log-in as administrator on Windows server 2.    stop all modules using the command safekit shutdown 3.    close all editors, file explorers, shells, or cmd under SAFE and SAFEVAR (to avoid package uninstallation error) (SAFE=C:\safekit if %SYSTEMDRIVE%=C: ; SAFEVAR=C:\safekit\var if %SYSTEMDRIVE%=C:) 4.    uninstall SafeKit using the Control Panel-Add/Remove Programs applet 5.    reboot the server 6.    delete the folder SAFE that is the installation directory of the previous install of SafeKit 7.    undo all configurations that you have done for SafeKit boot/shutdown (see section 10.4) 8.    undo all configurations that you have done for firewalls rules setting (see section 10.3) 9.    uninstall the SafeKit Web application if installed (see section 3.1.4.2) 10. since SafeKit 8.2.5, the following SafeKit drivers are uninstalled but are no longer automatically removed from the Windows Driver Store: ·         viplwf (Virtual IP load balancer) ·         ndis6pckt (ARP rerouter) ·         rfsfilter (File replication filter) If you want to remove them from the Driver store, follow the procedure below: a.    make sure all related driver services are stopped by executing the following commands: net stop rfsfilter net stop viplwf net stop ndis6pckt b.    list all third-party driver packages installed in the Windows driver store, using the command: pnputil /enum-drivers Example output: … Published Name:     **oem4.inf** Original Name:      **ndis6pckt.inf** Provider Name:      Evidian Class Name:         Network Protocol … Published Name:     **oem18.inf** Original Name:      **saferfs.inf** Provider Name:      Evidian Class Name:         FS Replication filters … Published Name:     **oem5.inf** Original Name:      **viplwf.inf** Provider Name:      Evidian Class Name:         Network Service … c.    identify the ‘Published name’ (oemxx.inf) corresponding to the SafeKit drivers Look for entries where the ‘Original Name’ matches the following: viplwf.inf, ndis6pckt.inf, and saferfs.inf. d.    delete each SafeKit driver using the following command: pnputil /delete-driver oemxx.inf /uninstall Replace oemxx.inf with the actual name identified in step c. ### 2.5.2         Uninstall on Linux as root 1.    open a Shell console as root on Linux server 2.    stop all modules using the command safekit shutdown 3.    close all editors, file explorers, shells, or terminal under SAFE and SAFEVAR (SAFE=/opt/safekit ; SAFEVAR=/var/safekit) 4.    uninstall SafeKit using the safekit uninstall -all command and answer yes when prompted to delete all SafeKit folders 5.    reboot the server 6.    undo all configurations that you have done for firewalls rules setting (see section 10.3) 7.    delete the user/group created by the previous install (default is safekit/safekit) with the commands:  userdel safekit groupdel safekit 8.    uninstall the SafeKit Web application if installed (see section 3.1.4.2) ## 2.6             SafeKit documentation | | | | --- | --- | | *SafeKit Solution* | The SafeKit solution is fully described. | | *SafeKit Training* | Refer to this online training for a quick start in using SafeKit. | | *SafeKit Release Notes* | It presents: ·         latest install instructions ·         major changes ·         restrictions and known problems ·         migration instructions | | *Software Release Bulletin* | Bulletin listing SafeKit 8.2 packages, with descriptions of changes and fixed issues. | | *SafeKit Knowledge Base* | List of known SafeKit issues and restrictions. Other KBs are available on the Support portal, but are only accessible to registered users. | | *SafeKit User's Guide* | This is the guide. Please refer to the guide corresponding to your SafeKit version number. It is delivered with the SafeKit package and can be accessed via the web console under /User’s guide. The link opposite takes you to the latest version of this guide. | # 3. The SafeKit web console ![*](safekituserguideen_fichiers/image001.png)       Section 3.1 “Start the web console” ![*](safekituserguideen_fichiers/image001.png)       Section 3.2 “Configure the cluster”   ![*](safekituserguideen_fichiers/image001.png)       Section 3.3 “Configure an application module” ![*](safekituserguideen_fichiers/image001.png)       Section 3.4 “Monitor an application module” ![*](safekituserguideen_fichiers/image001.png)       Section 3.5 “Snapshots or logs of application module for debug and support” ![*](safekituserguideen_fichiers/image001.png)       Section 3.6 “Secure access to the web console”   The SafeKit 8 web console and API have evolved from earlier versions. As a result, the console delivered with SafeKit 8 can only administer SafeKit 8 servers, which cannot be administered with an older console. ## 3.1             Start the web console The web console permits to administer one SafeKit cluster. A SafeKit cluster is a set of servers where SafeKit is installed and running. All servers belonging to a given SafeKit cluster share the same cluster configuration (list of servers and networks used) and communicate with each other’s to have a global view of SafeKit modules configurations. The same server can not belong to many SafeKit clusters. ### 3.1.1         Start a web browser The prerequisites for launching the console are as follows: ·         The device (workstation, server, or mobile) must have network access to the SafeKit server(s) and the necessary access permissions. ·         Networks, firewalls, and proxies must be configured to allow access to all SafeKit servers. ·         Supported browsers include Microsoft Edge, Firefox, and Chrome. Starting with version 5, the console is also available as a Progressive Web App (PWA) on both desktop and mobile devices (see section 3.1.4). | | | | --- | --- | | Commentaire, ajouter contour | The console uses a separate versioning system from the SafeKit package. | ### 3.1.2         Connect to a SafeKit node By default, access to the web console requires the user to authenticate himself with a name and password. On SafeKit install, you had to initialize it with the user admin and assign a password. This admin name and password are sufficient to access all the console's features. For more details on this configuration, see section 11.2.1. 1.    Start a web browser (Microsoft Edge, Firefox, or Chrome) 2.    Connect it to the URL http://host:9010 (where host is the name or IP address of one of the SafeKit servers). If HTTPS is configured, there is an automatic redirection to https://host:9453. 3.    The SafeKit server to which the console is connected (host in the URL) is called the **connection node**. This node acts as a proxy to communicate on behalf of the console with all other SafeKit servers. | | | | --- | --- | | Commentaire, ajouter contour | You can connect to any node of the cluster since the console offer global view and actions. On connection error with one node, connect to another node. | 4.    In the login page, enter admin as user’s name and the password you gave on initialization (e.g., pwd). 5.    The SafeKit web console is loaded ![](safekituserguideen_fichiers/image050.png) ·         When the console is connected to a SafeKit server on which the cluster is configured, the name of the node corresponding to the server (as defined in the cluster configuration) is displayed in the header. This is the **connection node** (node1 in the example). If the cluster is not yet configured, no name is displayed. ·         Click on ![](safekituserguideen_fichiers/image046.png) to open the menu to read the SafeKit User’s Guide, select the language, enable/disable the dark mode and logout. ·         Click on ![](safekituserguideen_fichiers/image051.png) to collapse or expand the navigation sidebar. ·         Click on ![](safekituserguideen_fichiers/image052.png) “Configuration” to configure the cluster and the modules. Configuration is only authorized to users that have Admin role. By default, the admin user has the Admin role. ·         (4) Click on ![](safekituserguideen_fichiers/image053.png) “Monitoring” to monitor and control the configured modules. Monitoring is authorized to users that have Admin, Control and Monitor roles. With Monitor role, actions on modules (start, stop…)  are prohibited. | | | | --- | --- | | Commentaire, ajouter contour | The web console offers contextual help by clicking on the icon. | ### 3.1.3         List of connection nodes The console offers the ability to easily switch connection node, even if the node belongs to a different cluster. ![](safekituserguideen_fichiers/image054.png) ·         (1) Click on![](safekituserguideen_fichiers/image055.jpg)to display the list of connection nodes. By default, if the cluster is configured, it lists all the nodes belonging to the cluster. Otherwise, this menu is unavailable. ·         (2) For example, click on node2 to connect the web console to node2. This is a shortcut for loading the web console from this node, which becomes the connection node. ·         (3) Click on ![](safekituserguideen_fichiers/image056.png) to open the submenu that allows to: o    Export the list of connection nodes to a JSON file on your workstation. It contains the names and addresses of the nodes in the list. o    Import the list of connection nodes from a JSON file on your workstation. Use the exported JSON file as a template and define your own list of connection nodes. This can be used if you wish to manage nodes belonging to different SafeKit clusters. You can also define a cluster name to which the nodes belong. o    Import the cluster nodes Use this option to reset an imported list and restore the default list, which contains only the nodes defined in the cluster of the node to which the console is connected. ### 3.1.4         Use the SafeKit web application Starting with version 5 of the SafeKit console, it is also available as a Progressive Web App (PWA) on both desktop and mobile devices. Using the SafeKit web application offers several advantages: ·         User Experience The console can be launched directly from the home screen, start menu, etc. This provides a user experience similar to that of a native application, especially on mobile devices. ·         Real-time notifications When enabled, notifications allow users to receive immediate updates about module state changes. o    On mobile devices, notifications are available only through the SafeKit web application. Additionally, HTTPS is required. o    On desktop platforms, notifications work in both the web application and the browser. However, when accessed remotely, HTTPS is required; HTTP is only allowed on localhost. ·         Enhanced security The web application benefits from isolation from browser tabs and uses the HTTPS protocol along with service workers for improved protection. To install the SafeKit web application, the platform and browser used must support PWAs. This includes browsers such as Chrome, Microsoft Edge, and Firefox. Once installed, the application becomes standalone and has its own uninstallation process. #### 3.1.4.1      Install the web application On desktop 1.    Load the SafeKit web console in the web browser using the URL https://:9453 | | | | --- | --- | | Commentaire, ajouter contour | On a SafeKit node, you can use http://localhost:9010. | 2.    Locate the installation icon in the browser’s address bar: | | | | --- | --- | | Microsoft Edge | It appears as a small icon next to the URL Or go to the menu Apps | | Chrome | It appears as a small icon next to the URL |   3.    Click to install then confirm The SafeKit console will then appear as a standalone application, with its own icon in the Start menu, on the desktop, etc. On mobile 1.    Load the SafeKit web console in the web browser using the URL https://:9453 2.    If your mobile device and browser support PWAs, look for the “Add” action | | | | --- | --- | | Chrome (Android) | In the menu icon (⋮) in the top-right corner |   3.    Select “Add to Home screen” then confirm The SafeKit console will be accessible from the home screen, just like a native application. #### 3.1.4.2      Uninstall the web application On most platforms, the web application is uninstalled in the same way as other applications. On some desktop operating systems, the application can be uninstalled directly from the opened application by accessing the menu located in the upper-right corner. ### 3.1.5         Update the web console Each upgrade of the SafeKit package may include a new version of the web console. To load this new version, it may be necessary to clear the browser cache. You can do this using a keyboard shortcut: 1.    Open the browser to any web page and hold CTRL and SHIFT while tapping the DELETE key 2.    A dialog box will open to clear the browser. Set it to clear everything and click Clear Now or Delete at the bottom 3.    Close the browser, stop all background processes that may be still running and re-open it fresh to reload the web console   Starting from version 5 of the SafeKit console, it includes support for Progressive Web Application (PWA) functionality. This enhancement introduces automatic update capabilities. However, if the automatic update does not trigger, you can force it manually: ·         Either by clearing the browser cache, as described above ·         Or by clicking the “Update” button available in the menu located in the top-right corner. If a new version is available, the console and its version will be updated accordingly. ![](safekituserguideen_fichiers/image059.jpg) ## 3.2             Configure the cluster The SafeKit cluster must be defined before installing, configuring, or starting a SafeKit module. A Safekit cluster is defined by a set of networks and the addresses, on these networks, of a group of SafeKit servers, named nodes. These nodes implement one or more modules. Each server is not necessarily connected to all the networks, but at least one. The cluster configuration is saved on the servers’ side into the cluster.xml file (see section 12). For a correct behavior, it is required to apply the same cluster configuration on all the nodes. | | | | --- | --- | | Commentaire important contour | You must fully define the cluster configuration before installing and configuring modules since the modification of the cluster can affect the configuration or the execution of installed modules. |   The cluster configuration home page is available: ·         Directly via the URL http://host:9010/console/en/configuration/cluster Or ·         By navigating the console via ![](safekituserguideen_fichiers/image061.png) “Configuration/Cluster configuration”   If the cluster is not yet configured, the cluster configuration wizard is automatically opened. ### 3.2.1         Cluster configuration wizard Open the configuration wizard: ·         Directly via the URL http://host:9010/console/en/configuration/cluster/config Or ·         Navigate in the console via ![](safekituserguideen_fichiers/image040.png)“Configuration/Cluster configuration/ ![](safekituserguideen_fichiers/image061.png)Configure the cluster/”   The cluster configuration wizard is a step-by-step guided form: ![SafeKit web console image The cluster configuration wizard](safekituserguideen_fichiers/image062.jpg) 1.    “Edit cluster configuration” described in section 3.2.1.1 2.    “Check result” described in section 3.2.1.2 3.    ![](safekituserguideen_fichiers/image063.png) to “Exit cluster configuration wizard” #### 3.2.1.1      Edit cluster configuration ![SafeKit web console image Edit the cluster configuration using the cluster configuration wizard. Define 2 LANs and IP addresses of 2 nodes on these networks ](safekituserguideen_fichiers/image064.png) ·         (1) Fill in the form to first assign a user-friendly name for the network. This name is used for configuring heartbeat networks used by a module. Click on ![](safekituserguideen_fichiers/image065.png) to add another node/lan or on ![](safekituserguideen_fichiers/image066.png) to remove the node/lan from the cluster. | | | | --- | --- | | Commentaire important contour | When a node/lan is removed from the cluster, all modules using it in its configuration may become unusable. | ·         (2) Fill in the IP address of the node and then press the Tab key to check the server connectivity and automatically insert the server hostname. The icon next to the address reflects the reachability of the node. | | | --- | | SafeKit web console image Tooltip with cluster node information in the cluster configuration wizard means that the SafeKit server is available. The tooltip gives information on the server. | | SafeKit web console image Tooltip for connection issues in the cluster configuration wizard means that there was no reply from the server within the timeout delay. Fix the problem to be able to administer this node. It may be a bad address, a network or host failure, a bad configuration of the web browser or the firewall, the stop of the SafeKit web service on the node. For solving the problem, refer to the section 7.1. |   ·         Change the node name if necessary. This name is the one that will be used by the SafeKit administration service for uniquely identifying a SafeKit node. It is also the one displayed into the SafeKit web console. ·         (3) If you prefer, click on “Advanced configuration” to switch to XML cluster editing. Click on ![](safekituserguideen_fichiers/image071.png) to open the SafeKit User’s Guide on the configuration description in the cluster.xml file. ·         Click on “Reload” to discard your current modifications and reload the original configuration. ·         (4) Once the edition is completed, click on “Save and Apply” to save and apply the edited configuration to all nodes in the cluster.   | | | | --- | --- | | Commentaire important contour | If required, you can reapply the configuration to all nodes without modifying it. |   | | | | --- | --- | | Sous-titres contour | For examples of cluster configurations with two networks  refer to section 15.1.1; with three nodes refer to section 15.2.1. |   #### 3.2.1.2      Check result ![SafeKit web console image Check the cluster configuration results on all nodes using the cluster configuration wizard](safekituserguideen_fichiers/image074.png) ·         (1) Read the result of the operation on each node: o    “Success”![](safekituserguideen_fichiers/image075.png) means the configuration was successful. o    “Failure”![](safekituserguideen_fichiers/image076.png) means the configuration has failed. Click to read the output of commands executed on the node and search for the error.  You may need to modify the parameters entered or connect to the node to correct the problem. Once the error has been corrected, “Save and apply” again. ·         (2) Click on “Configure modules” to exit the cluster configuration wizard and navigate to modules configuration. Or ·         (3) Click on ![](safekituserguideen_fichiers/image077.png) to “Exit the cluster configuration wizard” and navigate to the cluster configuration home page ### 3.2.2         Cluster configuration home page When the cluster is configured, the cluster configuration home page is available. Open it: ·         Directly via the URL http://host:9010/console/en/configuration/cluster Or ·         By navigating the console via ![](safekituserguideen_fichiers/image078.png)“Configuration/Cluster configuration”   In this example, the console is loaded from 10.0.0.107, which corresponds to node1 in the existing cluster. This is the connection node. ![SafeKit web console image Cluster configuration home page](safekituserguideen_fichiers/image079.png) ·         (1) Click on ![](safekituserguideen_fichiers/image080.png)“Configuration” in the navigation sidebar ·         (2) Click on “Cluster configuration” tab Nodes configured in the cluster are listed with their configuration date. ·         (3) Click on ![](safekituserguideen_fichiers/image081.png) to display details about the node: networks name and addresses defined in the cluster configuration, SafeKit version, license key, hostname, OS. ·         (4) Click on one of the buttons: o    ![](safekituserguideen_fichiers/image082.png) to modify the cluster configuration and/or re-apply it. This opens the cluster configuration wizard and loads the cluster configuration from the connection node. o    ![](safekituserguideen_fichiers/image083.png) to download the cluster configuration in XML format from the connection node. o    ![](safekituserguideen_fichiers/image084.png) to unconfigure the cluster on one or more nodes ## 3.3             Configure an application module Once the cluster has been set up, you can configure a new module on the cluster. The module configuration home page is accessible: ·         Directly via the URL http://host:9010/console/en/configuration/modules Or ·         By navigating the console via ![](safekituserguideen_fichiers/image085.png)“Configuration/Modules configuration” If no module has been configured, the console automatically presents the page for configuring a “New module”. | | | | --- | --- | | Sous-titres contour | For module configuration examples refer to section 15. | ### 3.3.1         Select the new module to configure In this example, the console is loaded from 10.0.0.107, which corresponds to node1 in the existing cluster. This is the connection node. ![SafeKit web console image Select a new module to configure from the configuration home page](safekituserguideen_fichiers/image087.png) ·         (1) Click on ![](safekituserguideen_fichiers/image088.png)“Configuration” in the navigation sidebar ·         (2) Click on “Modules configuration” tab ·         (3) Click on “New Module” The page proposes to select a new module among several proposals visible by clicking on ![](safekituserguideen_fichiers/image081.png): o    the “Main modules”, including the generic mirror.safe (refer to section 15.1.2) and farm.safe (refer to section 15.2.2) modules for integrating a new application into a mirror or farm architecture. Here are the modules stored on the connection node, node1, under SAFE/Application\_Modules/generic, SAFE/Application\_Modules/demo and SAFE/Application\_Modules/published. o    “Backup modules” archived on the connection node, which are saved when a module is uninstalled on this node. They are loaded from node1 under SAFE/Application\_Modules/backup. o    “Other modules” which are examples of SafeKit features used in modules supplied for testing purposes only. Refer to section 15 for the description some of them. They are loaded from node1 under SAFE/Application\_Modules/other. o    A locally stored module accessible from “Upload module”. This feature can be used to configure a module for a given application (e.g., Microsoft SQL Server, PostgreSQL…) downloaded from one of the SafeKit quick installation guides. ·         (4) Select a module to configure from those listed above. In the example, mirror.safe. ·         (5) Click on the button ![](safekituserguideen_fichiers/image088.png)Configure the new module. ·         A dialog opens to give the new module name ![SafeKit web console image Enter the name of the new module](safekituserguideen_fichiers/image089.png) ·         (6) Enter the name of the new module. ·         (7) Click on “Confirm” The module configuration wizard is opened. This is described below. ### 3.3.2         Module configuration wizard The module configuration wizard is a step-by-step guided form.: ![SafeKit web console image The module configuration wizard](safekituserguideen_fichiers/image090.png) 1.    “Edit module configuration” described in section 3.3.2.1 2.    “Edit module scripts (Optional)” described in section 3.3.2.2 3.    “Enable communication encryption (Optional)” described in section 3.3.2.3 4.    “Save and apply” described in section 3.3.2.4 5.    “Check result” described in section 3.3.2.5 6.    ![](safekituserguideen_fichiers/image091.png) to “Exit module configuration wizard” Note that module reconfiguration can only be applied to nodes on which the module in question is not started. Therefore, stop the module before starting the configuration wizard. | | | | --- | --- | | Commentaire important contour | If needed, you can reapply the module configuration on all nodes without modifying it. | #### 3.3.2.1      Edit module configuration Below is an example of editing the mirror.safe module configuration. ![SafeKit web console image Edit the module configuration using the module configuration wizard](safekituserguideen_fichiers/image093.png) ·         (1) Fill in the form to assign values to the various components, add or remove them. Click on ![](safekituserguideen_fichiers/image081.png) to open the detailed panel for each component. This form is used to enter only the main module configuration parameters. | | | | --- | --- | | Commentaire, ajouter contour | The names of the “Heartbeat networks” proposed are the names of the lans entered during cluster configuration. | ·         (2) For advanced module configuration, exhaustive compared to the form, click on “Advanced configuration”. This switches to editing the module configuration file in XML format, userconfig.xml. Click on ![](safekituserguideen_fichiers/image071.png) to open the SafeKit User’s Guide describing the configuration of the various components in the userconfig.xml file. ·         If necessary, click on “Reload” to discard your modifications and reload the complete original configuration (including scripts if these were modified in the next step). ·         (3) Once you have finished editing the module configuration, click on “Next step”.   | | | | --- | --- | | Sous-titres contour | For examples of mirror module configuration, refer to section 15.1.2 ; of farm module configuration, refer to section 15.2.2. | #### 3.3.2.2      Edit module scripts Below is an example of editing the mirror.safe module scripts. ![SafeKit web console image Edit the module scripts using the module configuration wizard](safekituserguideen_fichiers/image094.png) ·         (1) Click on “start\_prim” or “stop\_prim” to edit it and insert your application start/stop. Click on ![](safekituserguideen_fichiers/image095.png) to copy the content and edit it with your favorite syntax editor. Once done, paste the modified content into the input field with ![](safekituserguideen_fichiers/image096.png). ·         (2) If necessary, click on “Advanced configuration” to list the other module’s scripts and edit them (prestart, poststop, scripts for checkers…). ·         Click on ![](safekituserguideen_fichiers/image071.png) to open the SafeKit User’s Guide describing the module scripts. ·         If necessary, click on “Reload” to discard your modifications and reload the complete original configuration (including the module configuration if it was modified in the previous step). ·         (3) Once you have finished editing the module scripts, click on “Next step”.   | | | | --- | --- | | Sous-titres contour | For examples of mirror module scripts, refer to section 15.1.3 ; of farm module scripts, refer to section 15.2.3. | #### 3.3.2.3      Enable communication encryption Encryption of internal module communications between cluster nodes is enabled by default. For details, see section 10.7. ![SafeKit web console image Enable/disable encryption of module communications using the module configuration wizard](safekituserguideen_fichiers/image098.png) ·         (1) Click “Enable” to enable or disable encryption of module communications. | | | | --- | --- | | Commentaire important contour | When the module's encryption key is not identical on all nodes, internal communication is impossible. The configuration must be reapplied to all nodes to propagate the same key. |   To generate new encryption keys, you need to: 1.    disable encryption, then “Save and apply” configuration to all nodes 2.    enable encryption, then “Save and apply” configuration to all nodes ·         If necessary, click on “Reload” to discard your modifications and reload the complete original configuration (including the module configuration and scripts if these were modified in the previous steps). ·         (2) Once this step is complete, click on “Next step”. #### 3.3.2.4      Save and apply Step to select the nodes affected by the configuration. ![SafeKit web console image Select the nodes on which to save and apply the module configuration using the module configuration wizard](safekituserguideen_fichiers/image100.png) ·         (1) Check/uncheck to select/unselect nodes. Please note that the connection node (node1 in the example) is mandatory. There are 2 cases where “Save and Apply” is disabled: | | | --- | | SafeKit web console image Situation where the node cannot be selected, in the module configuration wizard, because the module is running The module on the selected node is started and, in a state, other than STOP (NotReady). | | SafeKit web console image Situation where the node cannot be selected, iin the module configuration wizard, because the node is not responding There was no reply from the node within the timeout delay. It may be a bad address, a network or host failure, a bad configuration of the web browser or the firewall, the stop of the SafeKit web service on the node. For solving the problem, refer to the section 7.1. | In both cases, uncheck the node or click on “Save and check” to apply it later, after stopping the module or solving the communication problem. ·         (2) Click on “Save and check” to save the edited configuration on the connection node and check its consistency. It then proceeds to the next step to display the result of this operation. Once this operation has been completed, any changes are saved on the connection node. The configuration wizard can be closed and relaunched later to apply the saved configuration. Until the saved configuration is applied, the last applied configuration of the module remains active. ·         (3) Click on “Save and apply” to save and apply the edited configuration on selected nodes. It then proceeds to the next step to display the result of this operation. If this operation is successful, the applied configuration becomes the active one for the module. | | | | --- | --- | | Commentaire important contour | On the server side, the module configuration is saved under SAFE/modules/*AM* (where *AM* is the module name). When reconfiguring a module, this directory is deleted and overwritten with the changes made in the console. Thus, on the servers’ side, you must close all editors, file explorers, shells or cmd under SAFE/modules/*AM* before applying the configuration (otherwise there is a risk that the apply fails). | #### 3.3.2.5      Check result The example below shows the result of the “Save and Apply” operation. The layout for “Save and Verify” is similar. ![SafeKit web console image Check the module configuration results on the nodes using the module configuration wizard](safekituserguideen_fichiers/image104.png) ·         (1) Read the result of the operation on each node: o    “Success”![](safekituserguideen_fichiers/image105.png) means the operation was successful. o    “Failure”![](safekituserguideen_fichiers/image106.png) means the operation has failed. Click to read the output of commands executed on the node and search for the error.  You may need to modify the parameters entered or connect to the node to correct the problem. Once the error has been corrected, repeat the operation from the previous step. ·         (2) Click on “Monitor modules” to exit the module configuration wizard and navigate to modules monitoring. Or ·         (3) Click on ![](safekituserguideen_fichiers/image077.png) to “Exit the module configuration wizard” and navigate to the modules configuration home page. ### 3.3.3         Modules configuration home page Once the first module has been configured, the module configuration home page is available. It allows you to view the modules installed on the cluster and to access the configuration of a new module. Open it: ·         Directly via the URL http://host:9010/console/en/configuration/modules Or ·         By navigating the console via ![](safekituserguideen_fichiers/image085.png)“Configuration/Modules configuration”   | | | | --- | --- | | Commentaire important contour | Before each reconfiguration, deconfiguration and uninstallation, on each node, close all editors, file explorer, shells or cmd under SAFE/modules/*AM* (or risk the operation failing). |   In the following example, the console is loaded from 10.0.0.107, which corresponds to node1 in the existing cluster. This is the connection node. ![SafeKit web console image Installed modules on the configuration home page](safekituserguideen_fichiers/image108.png) ·         (1) Click on ![](safekituserguideen_fichiers/image088.png)“Configuration” in the navigation sidebar. ·         (2) Click on “Modules configuration” tab. ·         Modules installed on the cluster are listed with the date the configuration was applied and, if applicable, the date the configuration was saved but not yet applied. ·         (3) Click on one of the buttons associated with the module: o    ![](safekituserguideen_fichiers/image109.png) to modify its configuration or reapply its current configuration. This opens the module configuration wizard and loads its current configuration from the connection node. o    ![](safekituserguideen_fichiers/image110.png) to download the .safe, consisting of all module files (userconfig.xml, scripts) from the connection node. o    ![](safekituserguideen_fichiers/image111.png) to reconfigure the module from the contents of a locally stored .safe. o    ![](safekituserguideen_fichiers/image112.png) to restore a previous module configuration. o    SafeKit keeps a copy of the last three successful configurations (stored under SAFE/modules/lastconfig on the server side). All module configuration files are packaged in a .safe file, whose name is of the type of *AM*\_\_