Best Practice: System Configuration Post Go-Live

Managing Change in the Production Environment

During implementation, you will likely be working with a consultant from Alemba to configure and deliver the ASM application. It's reasonable to assume you would be making these initial configurations in Production, but what happens after it has been delivered and your organization is using it on a day-to-day basis? How will you continue to keep the system updated and relevant, or even safely add new capabilities?

Audience: System Administrators/ASM Developers with sufficient rights to update, change, and configure ASM, ASM Screens, and ASM Workflows.

System Change and Configuration Options

Customers have two options after they take delivery of the ASM Application for how they will manage changes to the ASM application. Our recommended approach follows Option 1.

Option 1 - Develop/Sys Admin in Production and use Test solely for UAT.
  1. When ready for UAT, we simply take a copy of Production and refresh Test with it.

  2. All test data remains in the Test System

Most of Alemba's clients choose this option and it is also how we manage changes in our own system. They prefer to continue to make necessary changes and perform new work (workflows and screens) in Production, reserving the Test environment for User Acceptance Testing.

Role-Based Access Security Makes System Admin in Production Possible

The idea of developing in production is foreign and a bit scary to many organizations. However, ASM's Role Based Access Security makes it a snap to do and minimizes risk almost entirely.

Option 2 - Develop/Sys Admin in Test and Promote to Production

This method uses Configuration Portability (Config Port)

  1. We refresh Test with Production exactly as above.

  2. Developers/Sys Admins make changes

  3. The Change(s) are tested

  4. The Change(s) are exported from Test and Imported to Production

  5. All test data remains in the Test System

Some clients implement the Software Development Lifecycle (SDLC), making changes in test and promoting to production. However this method requires strict adherence to a change freeze in production that many organizations have difficulty to universally enforce. If you implement this methodology, no changes can be made in production apart from:

  • Call and Request Data

  • Person Records/Details

  • AI Ops Rules

  • System default values (such as the default System Title defined in System Administration)

  • IPK Workflow Rules

  • CMDB Items and Services, Service Actions

  • Forum Management Rules

  • Activity Logs

  • SLM Agreements

  • Passwords and Email settings

  • System customizations, stored procedures, and integration settings.

**The main issue organizations face is that simple, unauthorized changes cause Production and Test to fall out of Sync.

Why is it so important the databases be in Sync? If you have made changes in both Production and Test, the GUIDs needed for the Config Port utility to match records at the database level will not be aligned and the import will either fail entirely, or it will duplicate and confabulate the data in Production.

To learn more about Config Port or to use this option, please see: Configuration Portability (Config Port)

Immediate vs. Delayed Effect Changes

All changes to the production version of ASM have the potential to appear to the user's and analyst's with immediate effect. Many times we make immediate changes when the change is small and does not require testing or where UAT is not possible or may not even be necessary. Even with these types of changes, you should still log a Change Request if you are practicing Change Management.

Standard Changes that typically can be made in Production that do not need testing - These are a few examples (there can be many more):

  1. You have been asked to correct the spelling or change the text of a label on a screen.

  2. A workflow task is assigned incorrectly

  3. You need to add Analysts to groups

  4. An SLA needs additional criteria added, it needs a new expiration date, or renewal

  5. Your organizations Business Hours change and you need to update the Hours Matrix

Delayed Effect Changes - Concealing Changes from Users and Analysts

Where possible, you will want to make changes only in PRD, test the changes in UAT, and then get approval prior to making the changes visible to the end users (Enablement) in Production. How can you begin working on a change in the Production system without exposing it to the Users or Analysts until it is ready? Make use of ASM's RBAC and other security features.

To do this, you will generally create special IPK Groups, Workflow and Approval Groups, special Subscriber Groups and Security Roles. The groups and their related RBAC and settings will effectively allow you to wall-off your changes until such time as you are ready to enable them. We will cover this in more detail later.

Changes That Can Be Isolated From Users and Analysts

  • Screen Changes

  • New Screens

  • Message Template Changes

  • New Message Templates

  • Workflow Changes

  • New Workflows

Workflows and screens tend to be large and complex and we wouldn't want to manually set them all up in both environments. In this case, we will do the build in Production and then copy PRD down to Test for UAT. We will simply shield it from view in PRD until such time as it has passed UAT.

If QA rejects the build, you will get your list of defects, fix them, and then copy a fresh image of production back to UAT for the next test cycle that includes your fixes.

UAT issues are fixed in the source, Production, and a fresh image is deployed to Test.

When to Break the 'Rules'

Few things are ever totally black and white. There is often a grey area and developing in ASM is no different. Some changes, particularly those that impact the Portal Skins or behavior, or integrations should be made in UAT when there is no test cycle running (e.g., when UAT is dormant) and the change validated against any unintended consequences.

For example:

  1. Portal Skin and Styling Changes cannot be hidden from your end users unless:

    • You have created a test Portal on the Production system

    • Only Developers have access to that portal

    • You are not changing and global style sheets or default image files in the file structure of ASM (changes in the GUI are fine)

**Changes to global style sheets or default image files in the file structure of ASM will be immediately available and should be avoided in production until they have been tested.

  1. Service Catalog Changes unless:

    • You have created a test Portal on the Production system

    • Only Developers have access to that portal and the Service Actions being changed, e.g., you are not changing "Live Actions".

  2. Service Level Management Agreements unless:

    • The Agreement is inactive AND

    • You do not intend to test it before making it active

  3. Message Type Maps (Setting which email goes out and when)

    • Some small changes probably can be made without testing, such as Message Subjects or updating templates to message types, however

    • A major overhaul should be accomplished in the Test System First

  4. Subscriber Groups unless:

    • You are merely maintaining an existing group such as

    • Adding or removing subscribers

  5. Federated CMDB actions (this does not include general and normal maintenance and record keeping)

  6. Integrations, mapping, sources, etc

  7. Critical or Global System Administration Settings such as:

    • System Security and Protocols (e.g., enabling SSO)

    • Partitioning

    • Screens and Template purges

    • CMDB Item Type Cleanup

When you are sure of the safety and the intended result, you can make the immediate change in Production.

When in Doubt - Test it Out!

Imagine if you changed an integration in production and suddenly wiped clean thousands of records? Better to make that mistake in the UAT environment. If you blow up the Test system, you can simply have it restored.

Typically, its not too much trouble to make the changes listed above in UAT to test them before then going to production and recreating the settings manually once you are fully aware of the consequences and validity of the change..

Last updated