---
canonical: https://safekit.evidian.com/wp-content/uploads/downloads_safekit/version-82/safekituserguidehtml/documentation/safekituserguideen.htm
---

## 3.4             Monitor an application module

Once a module is configured, you can
monitor its state and run actions on it (start, stop…).

The modules monitoring home page is
accessible:

·        
Directly via http://host:9010/console/en/monitoring

Or

·        
By navigating the console via ![](safekituserguideen_fichiers/image119.png)“Monitoring”

### 3.4.1         Monitoring home page

|  |
| --- |
| In this example, the console is loaded from 10.0.0.107, which corresponds to node1 in the existing cluster. This is the connection node. Two modules are configured: farm and mirror.  SafeKit web console image Modules monitoring home page  ·         (1) Click on “Monitoring” in the navigation sidebar  For each installed module, it displays:  o    the module name and nodes name on which it is installed  o    the module state on the node  o    a notification on state change if the user has allowed them, and the URL is https or http://localhost  For a description, see section 3.4.2.  ·         (2) Click on to open the menu of global actions (start, stop…) on the module that apply on all nodes (node1, node2 in the example).  For a description, see section 3.4.3.1.  ·         (3) Click on to open menu of actions (start, stop…) on the module that applies only to the node (node1 in the example).  For a description, see section 3.4.3.2.  ·         (4) Click on the node panel (mirror>node1 in the example) to open details for the module on this node (logs, resources…). Since SafeKit 8.2.2, Click instead on Une image contenant noir, obscurité  Description générée automatiquement to open/close the details.  For a description, see section 3.4.4.  ·         (5) Click on Une image contenant noir, obscurité  Description générée automatiquement to open/close the module states timeline on all nodes where it is installed. Available since SafeKit 8.2.2.  For a description, see section 3.4.5. |

### 3.4.2         Module state

The module is represented real-time display
of its synthetic and detailed states on the left and right panels.

#### 3.4.2.1      Synthetic state

The console displays one of the following
synthetic states for the module on the node:

|  |
| --- |
| STOP (NotReady)(red)                    Module stopped (ready for starting) |
| WAIT (Transient)(orange)             Transient state of the module |
| ALONE (Transient)(orange)       Transient state of a mirror module, primary without secondary |
| ALONE (Ready)(green)                   Stable state of a mirror module, primary without secondary |
| PRIM (Transient)(orange)             Transient state of a mirror module, primary with secondary |
| PRIM (Ready)(green)                     Stable state of a mirror module, primary with secondary |
| SECOND (Transient)(orange)          Transient state of a mirror module, secondary            with primary, during the synchronization of   replicated directories |
| SECOND (Ready)(green)                  Stable state of a mirror module, secondary with primary |
| UP (Transient)(orange)                 Transient state of a farm module |
| UP (Ready)(green)                        Stable state of a farm module |
| WAIT (NotReady) (red)                   Blocked state of the module, waiting for one or more resources |
| NOT CONFIGURED (grey)                    Installed module but not configured |
| ERROR (red)                                                  The node did not respond within the given time limit.  This may be due to an incorrect address, a network or server failure, a misconfigured web browser or firewall, or the SafeKit web service being stopped on the node (see section 7.1). It may also be due to the temporary unavailability of the connection node. In this case, reload the console from another SafeKit node. |

 

For details on state changes of a mirror
module, see section 5.2.

For details on state changes of a farm
module, see section 6.2.

#### 3.4.2.2      Detailed state

It is the state of the main resources or
failover rules.

|  |
| --- |
| uptodate             Replicated directories of the mirror module are uptodate |
| Replicated directories of the mirror module are not uptodate  not uptodate |
| The mirror module is in degraded mode described in section 7.7   degraded |
| 50%, 100%             The network load share of the farm module (e.g. 50% or 100% with 2 nodes) |
| No load share taken by the farm module     0% |
| The module applied the failover rule (e.g., the rule named  c\_checkfile         c\_checkfile) which triggers the actions restart, stop, stopstart, or wait on the module due to a resource going down. View section 13.19.4.2 for details on failover rules. To analyze the issue, read the logs and resource statuses as described below. |
| The module is in state ERROR (red)  connection       The node did not respond within the given time limit                                                        error |

### 3.4.3         Module control menus

#### 3.4.3.1      Global menu

The actions of global menu apply to all
nodes where the module is configured.

In the example below, actions apply to the module mirror on node1 and node2.

![SafeKit web console image
Start or stop the module on all nodes, download all module snapshots or logs, using the global menu of the module
](safekituserguideen_fichiers/image136.png)

·        
(1) Click on ![](safekituserguideen_fichiers/image137.png)to open the
module's global actions menu.

·        
Click on “Start”
to start the module on all nodes.

For
mirror module, the node with the up-to-date replicated data is started as
primary.

·        
Click on “Stop”
to stop the module on all nodes.

For
mirror module, the node that is secondary is stopped first to avoid unnecessary
failover.

·        
Click on “Debug”
for debug and support as described in section 3.5.

#### 3.4.3.2      Local menu

The actions of local menu apply only to the
selected node.

##### 3.4.3.2.1                Control a mirror module

In the example below, actions apply to the module mirror on node1.

![SafeKit web console image
Actions using the local menu of the mirror module on the selected  node:
start, stop or restart  the module
force start as primary, as secondary, as secondary with full data synchronization](safekituserguideen_fichiers/image138.png)

·        
(1) Click on ![](safekituserguideen_fichiers/image137.png)to open module's
local actions menu on the desired node (e.g. node1).

·        
Click on “Start”
to start the module on the node.

For
mirror module, the node is started as primary when replicated data are up to date.
Otherwise, it is started as secondary. For details, see section 5.5.

·        
Click on “Stop”
to stop the module on the node.

·        
Click on “Restart”
to restart the module on the node.

It only
executes only stop then start scripts to locally restart the application
without leading to a failover.

·        
Use “Force start” submenu when you need to decide if the node should
start primary or secondary:

o    Select “Force start As Primary” to force the module to start as primary on this node.

For
instance, on the 1st start of a mirror module as described in section 5.3, you must “Force
start As primary” the node which has the
up-to-date replicated folders.

o    Select “Force Start As secondary” to force the module to start as secondary on this
node.

Data
synchronization can be optimized based on the module's last internal state.

o    Select “Force Start As secondary with full
data synchronization” to start the
module on this node as a secondary and to force a complete copy of the
replicated data.

·        
Click on “Disable/enable”
to control error detection as described in section 3.4.3.2.3.

·        
Click on “Debug” to download module logs or snapshots from this node
rather than from all nodes as described in section 3.5.

 

To understand and check the correct
behavior of a mirror module, see section 5. To test it, see section 4.

##### 3.4.3.2.2                Control farm module

In the example below, actions apply to the module farm on node2.

![SafeKit web console image
Actions using the local menu of the farm module on the selected  node:
start, stop or restart  the module](safekituserguideen_fichiers/image139.png)

·        
(1) Click on ![](safekituserguideen_fichiers/image137.png)to open
module's local actions menu on the desired node (e.g. node2).

·        
Click on “Start”
to start the module on the node.

·        
Click on “Stop”
to stop the module on the node.

·        
Click on “Restart”
to restart the module on the node.

It only
executes only stop then start scripts to restart the application without
leading to a failover.

·        
Click on “Disable/enable”
to control error detection as described in section 3.4.3.2.3.

·        
Click on “Debug” to download module logs or snapshots from this node
rather than from all nodes as described in section 3.5.

 

To understand and check the correct
behavior of a farm module, see section 6. To continue the
tests, see section 4.

##### 3.4.3.2.3                Control checkers or processes/services monitoring

To avoid false error
detection and automatic failover on application maintenance, you can disable
configured checkers (TCP, ping, custom….) or processes/services monitoring.
Once the maintenance is completed, they can be safely re-enabled. These actions
can be applied while the module is started/stopped and are not reset when the
module stops-starts.

In the example below, actions apply to the module mirror on node1.

![SafeKit web console image
Actions using the local menu of the module on the selected  node:
disable or enable configured checkers (TCP, ping, custom….) ; disable or enable configured processes monitoring and services monitoring](safekituserguideen_fichiers/image140.png)

·        
(1) Click on ![](safekituserguideen_fichiers/image141.png)to open the
module's local actions menu on the desired node (e.g. node1).

·        
(2) Click on “Disable/enable” to open the submenu.

·        
(3) Click on “Checkers”
or “Processes/services monitoring” to open the
submenu.

·        
(4) Click on “Disable”
to disable the error detection

This
disables all checkers (TCP, ping,
custom….) or processes/services monitoring configured
for the module.

·        
(4) Click on “Enable”
to re-enable error detection by checkers or processes/services monitoring.

### 3.4.4         Module details

You can display details for a module on one
node:

·        
Directly via the URL http://host:9010/console/en/monitoring /modules/*AM*/nodes/*node* (replace *AM* by the module name and *node* by the node name)

Or

·        
By navigating the console via ![](safekituserguideen_fichiers/image119.png)“Monitoring/Click on ![Une image contenant noir, obscurité

Description générée automatiquement](safekituserguideen_fichiers/image142.png) for the module>node”

 

The selected module>node is highlighted
with a blue color.

 

In the example, the detail for the module mirror on node1 is
displayed.

![SafeKit web console image
Click open or close details for the module on one node:
module logs with Logs tab
module resources with Resources tab
node information with Information tab](safekituserguideen_fichiers/image143.png)

·        
Click on ![Une image contenant noir, obscurité

Description générée automatiquement](safekituserguideen_fichiers/image144.png) to
open/close details for the module on this node (logs, resources…).

·        
Click on “Logs” tab to visualize the module logs.

·        
Click on “Resources” tab to visualize the module resources.

·        
Click on “Information” tab to visualize information on the node: networks name and addresses defined in the cluster
configuration, SafeKit version, license key, hostname, OS.

#### 3.4.4.1      Module logs

You can display logs of a module on one
node:

·        
Directly via the URL http://host:9010/console/en/monitoring /modules/*AM*/nodes/*node*/logs (replace *AM* by the module name and *node* by the node name)

Or

·        
By navigating the console via ![](safekituserguideen_fichiers/image121.png)“Monitoring/Click on the module>node/Logs tab”

 

The left panel displays in real-time the
non verbose module log for the selected module>node.

![SafeKit web console image
Display, filter, and download module logs using the Logs tab](safekituserguideen_fichiers/image145.png)

·        
Click on ![](safekituserguideen_fichiers/image146.png) to
resume/suspend the view in real time of the module log.

Refer to section 7 for an explanation of main messages.

·        
Click on ![](safekituserguideen_fichiers/image147.png) to download the
module log (verbose or not verbose).

·        
Select the message type to view:

|  |  |
| --- | --- |
| SafeKit web console image List of the different message levels in the module log and their associated icons, displayed in the Logs tab | ·         C(ritical) messages such as error detection  ·         E(vent) messages such as local and remote states  ·         U(ser) messages when the user run action on the module  ·         S(cript) messages when module scripts are executed |

·        
Click on a message to display the verbose module
log or the script log (output of scripts) into the log detail into the right
panel.

##### 3.4.4.1.1                Script log

To display the script log, click on the ![](safekituserguideen_fichiers/image088.png)S(cript) message whose output
you want to view.

![SafeKit web console image
Display the content of the script log by clicking on a S (Script) message in the module log using the Logs tab](safekituserguideen_fichiers/image149.png)

·        
(1) Click the ![](safekituserguideen_fichiers/image088.png)S(cript) message
consisting of:

o    the date and time of the execution of the script

o    the name of the script executed

o    the name of the name of the corresponding userlog
file

The userlog file content
is displayed into the right panel. In the example, it is the content of the
file SAFEVAR/modules/*AM*/userlog\_2024-02-12T091410\_start\_prim.ulog
(where *AM* is the module name)

##### 3.4.4.1.2                Verbose log

To display the verbose module log, click on
a message other than ![](safekituserguideen_fichiers/image088.png)S(cript).

![SafeKit web console image
Display the verbose module log by clicking on a message other than S (Script) in the module log using the Logs tab](safekituserguideen_fichiers/image150.png)

·        
(1) Click the message consisting of:

o    the date and time of the event

o    the module message

·        
All verbose messages between the selected
message and the previous one in the table are displayed in the right-hand
panel.

#### 3.4.4.2      Module resources

You can display resources of a module on
one node:

·        
Directly via the URL http://host:9010/console/en/monitoring /modules/*AM*/nodes/*node*/resources (replace *AM* by the module name and *node* by the node name)

Or

·        
By navigating the console via ![](safekituserguideen_fichiers/image119.png)“Monitoring/Click on the module>node/Resources tab”

##### 3.4.4.2.1                Ressources state

The left panel displays in real-time the
current state of the resources for the selected module>node.

![SafeKit web console image
Display the current value of the module resources using the Resources tab](safekituserguideen_fichiers/image151.png)

·        
(1) Select the group of resources to view:

|  |  |
| --- | --- |
| SafeKit web console image List of the different resource groups to be displayed in the Resources tab | ·         Module status  Main resources, especially the ones of files replication for a mirror module  ·         Checkers  Ressources set by checkers  ·         File replication  File replication-specific resources that demonstrate synchronization progress  ·         All resources |

·        
Click on a resource to display its value over
time in the right panel. This history may be empty for some resources
(unassigned or cleaned).

Resource’s state is controlled by the
failover machine to trigger a failover on failures (see section 13.19).

##### 3.4.4.2.2                Resource’s state value history

To display a resource's value history,
click on the resource you're interested in.

![SafeKit web console image
Display the history of a resource's values by clicking on the desired resource in the Resources tab](safekituserguideen_fichiers/image153.png)

·        
(1) Click on the line consisting of:

o    the last date the resource was assigned

o    the name and category of the resource. The full resource name is
like *category*.*name* (custom.checkfile
in the example).

The history of resource values is displayed
in the right panel. In the example, this is the custom.checkfile
resource corresponding to a resource assigned by a custom checker.

### 3.4.5         Module states timeline

Since SafeKit 8.2.2, you can display the module
states timeline:

·        
By navigating the console via ![](safekituserguideen_fichiers/image121.png)“Monitoring/Click on ![Une image contenant noir, obscurité

Description générée automatiquement](safekituserguideen_fichiers/image154.png) for the module”

This provides a global view of the module's
state on the cluster. Be aware:

·        
that the clocks of the two nodes must be
synchronized for the mapping of state changes to be meaningful

·        
it displays a reverse timeline of the module
states on all nodes over time, by starting by the newest date.

![SafeKit web console image
Display the timeline of a module's states on all nodes](safekituserguideen_fichiers/image155.png)

·        
Click on ![Une image contenant noir, obscurité

Description générée automatiquement](safekituserguideen_fichiers/image156.png) to
open/close the timeline. The timeline displayed is the one available at the
time of loading.

·        
Click on ![Une image contenant noir, obscurité

Description générée automatiquement](safekituserguideen_fichiers/image157.png) to
refresh the timeline with the latest state changes.

·        
Click on a state change event to display the
module log for the node starting at this date

## 3.5             Snapshots or logs of application module for debug and support

When the problem is not easily
identifiable, it is recommended to download logs or snapshots of the module on
all nodes as described below. Snapshots allow an offline and in-depth analysis
of the module and node status as described in section 7.18. If this
analysis fails, create an incident via the Support
portal, making sure to attach the snapshots.

In the following example, the module mirror is
configured on node1 and node2. Note
that a snapshot can be downloaded in any state of the module.

![SafeKit web console image
Download the snpahosts of the module from all nodes](safekituserguideen_fichiers/image158.png)

·        
(1) Click on ![](safekituserguideen_fichiers/image141.png)to open the global
menu of the module.

·        
(2) Click on “Debug” to open the debug submenu.

·        
(3) Click on “Download
the snapshots” to create and download the snapshot of the module for
each node.

The web console
relies on the web browser's download settings to save the snapshot on the
workstation. Some browsers may ask confirmation to download many files and zip
files.

The snapshot
generation command generates a new dump and creates a .zip file containing the
last 3 dumps and the last 3 module configurations.

In this example,
it downloads 2 snapshots: snapshot\_node1\_mirror.zip and snapshot\_node2\_mirror.zip.

·        
Click on “Download the
logs” to download the module log (verbose or not) for each node.

·        
In case of file replication issues, click on “Generate the dump files” at
the time the problem occurs.

The dump contains the module logs and
information on the system and SafeKit state at the time of the dump. It is
generated on the server side into SAFEVAR/snapshot/modules/*AM*/dump\_AAAA\_MM\_DD\_hh\_mm\_ss.

|  |  |
| --- | --- |
| Commentaire important contour | Since SafeKit 8.2.4, the zips generated for snapshots are protected by the password safekit. This allows the snapshot to be received in its entirety when sent via email. |

## 3.6             Secure access to the web console

SafeKit offers
different security policies for the web console that are implemented by
modifying the SafeKit web service configuration. These configurations also
offer role management:

|  |  |
| --- | --- |
| Admin role | This role grants all administrative rights by allowing access to   Configuration and  Monitoring in the navigation sidebar |
| Control role | This role grants monitoring and control rights by allowing access only to Monitoring in the navigation sidebar |
| Monitor role | This role grants only monitoring rights, prohibiting actions on modules (start, stop…) in Monitoring in the navigation sidebar. |

SafeKit provides different setups for the web service to enhance the security
of the SafeKit web console. The predefined setups are listed below from least
secure to most secure:

·        
HTTP. Same role for all users without
authentication

This
solution can only be implemented only in HTTP and is not compatible with user
authentication methods. It is intended to be used for troubleshooting only.

·        
HTTP/HTTPS with user authentication based on
Apache files and optional role management

It relies
on Apache files to store username/password for authenticating users and,
optionally, to store the associated role for restricting their access. To
connect to the console, the user must enter the username and password as
configured with the Apache mechanisms.

This is the
default active configuration, applied for HTTP and initialized with a single admin user
with the Admin role. The default setup can be extended to add users or to
switch to HTTPS.

·        
HTTP/HTTPS with user authentication based on
LDAP/AD authentication. Optional role management

It relies on
LDAP/AD authentication server to authenticate users and, optionally, restricts
their access based on roles. To connect to the console, the user must enter the
username and password as configured into the LDAP/AD server. It supports HTTP
or HTTPS.

·        
HTTPS with user authentication based on OpenID
Connect authentication. Optional role management

It relies on
OpenID Identity Provider server to authenticate users and, optionally,
restricts their access based on roles. To connect to the console, the user must
enter the username and password as configured into the Identity Provider
server. Since SafeKit 8.2.3, it supports only HTTPS.

 

To implement them, refer to the section 11.