Configuration Management

Introduction

Configuration management is a general term used to define the process of making ongoing changes to a product while limiting the possibility of introducing problems or breakages.

General Concepts

Environment Segregation

To adopt a robust change management process, it is important to understand the lifecycle of a product and the changes being applied. The minimum requirement to achieve some level of change control is to have a distinct test environment, separate to the live environment. This allows whoever manages the test environment to make and test manual changes, then utilising the configuration management process, roll out the configuration to the other environment.

To achieve a more robust process, a common set of environments that we recommend are:

• DEV: Allows changes to be applied freely and in isolation in order to investigate and solve problems or meet requirements

• UAT: Allows changes to be rolled out in a controlled manner in order to be tested as part of the solution

• LIVE: Allows only approved changes from UAT to be applied in order to ensure minimal risk of disruption to production

Without configuration management, changes need to be manually applied to each environment. This introduces the risk of human error and its management requires more top-level administration users. Software features that allow a configuration to be mapped from one environment to another resolve this problem.

Version Control

Version control is critical to ensuring the configuration management process is robust. In essence, this means storing a copy of the configuration outside the application in some form of source control technology, such as Git.

Taking this approach ensures a full audit history of any changes, and where necessary, the ability to roll back specific revisions.

Autoform DM's configuration is exported as json data, which can be easily stored in version control if needed in order to provide a more robust configuration management / change control solution.

Continuous Deployment

Continuous deployment is the process of being able to continuously make and deploy changes across multiple environments with minimal manual interaction. It ensures any overheads for change rollout are minimized and encourages smaller, more frequent individual changes, rather than infrequent, large bundled rollouts which are more intrusive.

More details on full continuous deployment support will be contained in future Autoform DM updates; however, the implementation of configuration management is an important step towards this goal.

The following example shows the concept of continuous deployment, including user access, environment segregation and version control as core aspects of the overall solution.

Autoform DM Configuration Management

The Autoform DM implementation of configuration management is flexible enough for many ad-hoc use cases and will ultimately be leveraged to facilitate continuous deployment scenarios. A proficient understanding of the configuration in Autoform DM is recommended before attempting to use this feature.

Usage

There are two entry points into the configuration feature; the web-front end and the REST API. Both expose the same basic import/export functionality. Additionally, a validation feature is available to trigger a test import that simulates what would happen for the selected configuration file, without risk of applying changes and breaking anything.

The Autoform DM configuration management front-end screens are shown.

Export

Manual interaction will be common for exports as the UI allows for the individual selection of export items. There is also a screen for manual interaction with the import for convenience.

Import

If in doubt, we always recommend that users run Validate on their configuration before importing, to verify the report reflects the expected outcome.

Additionally, both export and import are exposed via the REST API because most automated interactions are likely to be performed using this method.

A purely manual process would be for a user to log in, select the items to export and produce a configuration.json file. Then log in to another system and upload that file, validate it to check for errors, then perform the import.

Alternatively, log in, select the items to export and produce a configuration.json file. Run this JSON file through an automated script, allowing it to be automatically deployed to the relevant target systems via the REST API.

Full automation using the /export endpoint is currently not recommended due to the complexity of building the export definition, however it is possible by traversing other resources.

Data Format

The data format used by the configuration management feature is JSON (Java Script Object Notation). This is a simple text-based format that is human-readable and supports version control. It also matches the format used by the REST API in Autoform DM. An example follows:

{
  "properties": {
    "dmVersion": "X.x",
    "applicationServer": "test-vm",
    "databaseServer": "test-vm/DM8NEW",
    "exportTimestamp": "2019-10-07T14:20:00.267+01:00"
  },
  "existingItemStrategy": "KEEP",
  "existingItemRelationshipStrategy": "REPLACE",
  "documentDefs": [
    {
      "name": "account-statement",
      "label": "Account Statement",
      "keyDefs": [
        "account-number",
        "customer-number"
      ],
      "properties": {}
    }
  ],
  "keyDefs": [
    {
      "name": "account-number",
      "label": "Account Number",
      "dataType": "STRING",
      "cardinality": "SINGLE",
      "databaseIdentifier": "SKEY1",
      "docDefs": [
        "account-statement"
      ]
    },
    {
      "name": "customer-number",
      "label": "Customer Number",
      "dataType": "STRING",
      "cardinality": "SINGLE",
      "databaseIdentifier": "SKEY2",
      "docDefs": [
        "account-statement"
      ]
    }
  ]
}

The example export contains:

• A properties section detailing the source of the export

• Some strategy flags (covered later in this page)

• An example document definition

• Two example key definitions

Item Hierarchy and Relationships

Most configurable items in Autoform DM conform to a hierarchy which dictates the dependencies between configurable items. This diagram shows the relationships:

The export screen is ordered and structured in such a way to reflect this (starting at the top). It is important to note that selecting an item will force selection of any items with which it has a Depend on relationship.

For example, selecting the document definition Account Statement will force selection of the key definition Account Number on which it depends. However, the inverse is not true – selecting the key definition Account Number would not force selection of the document definition Account Statement because this is only a References link. As a result, it is possible to export the Account Number without the Account Statement. Importing that into a target system would be successful if the Account Statement already exists, otherwise it would fail stating that a referenced item did not exist.

In summary, Depends On links are forced on export. References links are not forced on export, but will still fail on import if the link cannot be established due to a missing item.

This flexibility allows items with References links to be exported and updated without the items they reference necessarily needing to be updated as well, enabling partial rollout of changes as required.

Existing State Interaction

Importing data into a blank instance of Autoform DM is simple and conflict-free. Importing into a system with existing configuration raises the question of conflict resolution, both in terms of existing items and the relationships between them. This is where the two strategies come in.

Existing Item Strategy

Defines how the import process treats items which exist in the target system but not in the export.

The options for this are:

• KEEP : (Default) Items that exist in the target system but are not referenced in the export will not be affected.

• DELETE : Item groups that exist in the target system, but which are not included in the export, will be deleted.

In most cases KEEP is likely to be the desired action. However, when keeping systems aligned, it is usual to remove old, unused configuration elements and this is when the DELETE option would be suitable. An example scenario involving an application, a document definition and two key definitions follows:

Note

Green means created, blue means updated and red means deleted.

Existing Relationship Strategy

Defines how the import process treats existing item relationships that are not referenced in the export when updating items.

The options for this are:

• MERGE : Combines the relationships provided on an exported item with those on the matching item in the target system.

• REPLACE : (Default) Replaces the relationships on the existing items in the target system with those provided in the export.

In most cases REPLACE will help keep two systems aligned; however, when deploying smaller configuration packs which may share configuration, MERGE allows some flexibility in not overwriting existing relationships. An example follows:

Note

Green means created, blue means updated and grey means unchanged.

As opposed to Replace:

Note

Green means created and blue means updated.

Users

Users are an outlier in the world of configuration management as they are often environment-specific or handled by LDAP. As a result they come with some usage complications as detailed below.

We recommend managing user access via group membership and LDAP.

Strict deletion of users is still unsupported and, therefore, configuration management still handles 'deletion' as a hide/disable operation.

In order to provide flexibility in this area, an optional flag, Include User References, is available. In most cases this can be left as false, ignoring any direct user access to applications or groups. However, if direct user access is managed in Autoform DM and needs to be mirrored on the target system, selecting true will make sure it is respected.

To ensure security, Autoform DM does not export user passwords; however, the creation of users via import does require that a password is specified. Currently, users will be exported with a blank password field which must be populated manually with a temporary password prior to import. This applies if being used to create the user in the new system. That password will then expire on next login and the user will be able to specify a new one.

For example:

{
  ...
  "users": [{
    "user" : {
      "username": "john",
      "status": STANDARD,
      "accountExpiry": "2019-01-01T00:00:00.000Z",
      "emailAddress": "string",
      "noPasswordExpiry": true,
      "bypassLockout": false,
      "changePasswordNextLogin": true,
      "fullName": "John Smith",
      "roles": [
        "pdm", 
        "pdmAdmin"
      ],
      "groups": [
        "testGroup"
      ]
    },
   "password": "a$6hyYtq*&Av" // required by the import
  }]
  ...
}

Client Applications

Client applications consist of two parts: the metadata known to Autoform DM (name, version, user/group access etc) and the binary application itself. The configuration management feature is only concerned with metadata configuration and, therefore, cannot handle installation of client applications.

When a client application has been installed in an environment, its metadata can be updated or exported as part of the feature; however, future updates to the binary must be performed manually.

Documents

Documents are not classed as part of the configuration so it is important that the configuration management feature in no way manipulates or destroys document data. As a result, any import operation which would require the deletion of dependent documents, i.e. the removal of a key definition, will fail.

Configuration that relates directly to document data is:

• Key definitions

• Document definitions

• Document definition properties

• Users