Composr Supplementary: Staging Servers

Written by Chris Graham

Potentially Outdated Tutorial

This supplementary tutorial might be outdated as it was written for a previous version of Composr CMS (version 10).


When running a professional website where a team of stakeholders are involved you are likely to want a staging server that mirrors the live server, for testing before pushing a site live.

The staging server will be updated multiple times, as the site develops.

This tutorial provides a few ideas and suggestions on process.

Basic architecture

You will have the following machines running your code:
  • Live server(s)
  • Staging server
  • Any number of development machines

Installation options

You can code the _config.php file to have multiple configurations, serving individual machine settings. This allows you to share a single configuration file between sites.

For example, you may have this code:

Code (PHP)

<?php

/* Normal _config.php settings go here, not shown in this example */

// Per-machine overrides
switch (gethostname()) {
    case 'example': // live machine, uses the main settings from above
        break;

    case 'staging': // staging machine
        unset($SITE_INFO['base_url']); // Allow dynamic base URL

        // Different DB settings
        $SITE_INFO['db_site'] = 'cms';
        $SITE_INFO['db_site_host'] = 'localhost';
        $SITE_INFO['db_site_user'] = 'cms';
        $SITE_INFO['db_site_password'] = 'abcdef';
        break;

    case 'dev01': // dev machine
        unset($SITE_INFO['base_url']); // Allow dynamic base URL
        $SITE_INFO['backdoor_ip'] = '127.0.0.1'; // Automatic login for localhost
        $SITE_INFO['no_email_output'] = '1'; // Suppress any e-mails going out (the 'mail_queue_debug' configuration option would be another choice)

        // Different DB settings
        $SITE_INFO['db_site'] = 'cms';
        $SITE_INFO['db_site_host'] = 'localhost';
        $SITE_INFO['db_site_user'] = 'cms';
        $SITE_INFO['db_site_password'] = 'abcdef';
        break;

    default:
        exit('Unrecognised server, ' . gethostname());
}
 

Synching files

There are 2 plausible approaches for synching files:
  1. Using Git. You are likely already using Git for development. Consider also using the same Git repository for deployment, as described below. This is what we recommend.
  2. Zipping up changed files. Our bundled recentchanges.sh script can do this.

Using Git

Using Git works as follows:
  1. Install Git on the staging server and live server.
  2. Set up the staging and live server SSH keys as 'deploy keys' on your Git server (deploy keys are read-only keys).
  3. When you have finished development and developer testing, do a pull to the staging server, and conduct testing there, then when ready pull to the live server.

More sophisticated users may:
  1. Use multiple Git branches so you can parallel-develop new functionality while functionality is going through testing.
  2. Also run a continuous integration server, where changes made in Git (or multiple Git branches) have automated tests run against them every time a new push is made.

The level of sophistication you have will typically depend on the size of your development team.

It is easy to accidentally run Git as the wrong user, messing up file ownership. This .bashrc line will help you avoid that:

Code (Bash)

alias git='echo Stop using Git as the wrong user'
 

You may want to run git config core.fileMode false, as you are unlikely to want file permissions to be tracked.

There is a non-bundled addon called 'Git Status' which provides a nice UI to seeing what changes have been made on-server and off-server.

Make sure your web server configuration blocks access to the .git directory. The default Composr .htaccess file should do this for Apache users.

Synching data

Initially you can push a database live just by doing an SQL dump and import using standard MySQL tools. However, it becomes much more difficult to managing subsequent changes as your live server will have data that you cannot just overwrite with data from a development machine (you would lose live data and inject test data).

There are 3 broad kinds of data:
  1. Content (e.g. pages or galleries)
  2. Configuration (e.g. configuration options)
  3. Structure (e.g. catalogues)

Each of these kinds are covered in their own section below.

You may also wish to occasionally replace the staging site database with that of the live server, to improve the accuracy of your testing environment.

Content

There are 4 plausible approaches for synching Content:
  1. Make content on the live server, and use the Composr Validation feature to control it going live to regular users. This is what we recommend.
  2. Make it on the staging site and copy it live by manual copy and paste of the data into a new live entry.
  3. Make it on the staging site and copy it live by copying a pseudo-file using WebDAV. This is only recommended for organisations that can invest in tuning and testing Composr's WebDAV implementation to their need; it's easy once a good workflow and understand is in place, but it does have some complexities under-the-hood.
  4. Make it in code using the Composr API and run code to make it live. This is similar to the approach for Configuration, so is covered below.

Page content is a special case. Because Comcode pages are stored as files, you may manage these directly through Git. The 'Git Status' addon makes this easier – you can easily download staging server changes to a development server, revert them, then pull them through properly using Git.

Configuration and Structure

There are 3 plausible approaches for synching Configuration and Structure:
  1. Write scripts to do your changes and make them live. Recommended for developers.
    1. By putting the code in data_custom/execute_temp.php – this is a spot reserved for temporary custom code. This approaches makes sense if the developer is the one pushing code to live.
    2. By creating a site-specific versioning script. This is described in the section below. This is recommended for serious development.
    3. By noting down Commandr commands used to do the configuration, then re-executing them on live later. This is only really recommended for small occasionally changes, but for those it works well due to the interactivity of it.
  2. Note down every change you're making and manually reapply it live.
  3. Use diff tools to compare database dumps taken at different points in times, and manually transfer the database changes. This is not recommended due to complexity, but useful when in a tight spot.

Coding up structure programmatically

When coding up your scripts, you'll find we have defined a number of specialised functions in the API to make your life a lot easier. These functions cover the most common structural configuration for a site.

Functions:
  • Misc:
    • define_custom_field – create a custom field
    • mass_set_page_access – set page access
    • define_redirects – set Composr-level redirects
  • Menu structure:
    • import_menu_spreadsheet – import a menu structure
    • export_menu_spreadsheet – export a menu structure
    • create_menu_structure – create a menu structure programmatically
  • Comcode pages:
    • rename_live_comcode_page – fix a page's database references (renaming the .txt file may not be enough)
    • define_comcode_page_structure – define hierarchical relationships between pages
    • define_page_metadata – define metadata for a Comcode page, like custom fields
  • Translation (after being manually given string remappings by a human translator):
    • content_lang_string_translation – translate content programmatically
    • lang_string_translation – create language packs programmatically

Site-specific versioning scripts

You can give each site update an incremental version number, and code in changes relating to this update. By coding it within a Composr systems/startup hook it will run on the first page view as soon as Git code is pulled through.

Here's an example:

Code (PHP)

<?php

class Hook_startup_upgrading
{
    /**
     * Run startup code.
     */
    public function run()
    {
        $version_at = intval(get_value('pseudo_version', '0', true));

        $version_check = 1;
        if ($version_at < $version_check) {
            set_option('wysiwyg_font_units', 'px');

            require_code('caches3');
            erase_cached_templates();
        }

        $version_check = 2;
        if ($version_at < $version_check) {
            set_option('enable_previews', '1');

            require_code('caches3');
            erase_cached_templates();
        }

        // ^ Add a new code block, with an incremented version number, for each targeted updated

        if ($version_at != $version_check) {
            set_value('pseudo_version', strval($version_check), true);
        }
    }
}
 

I this example we have coded in 2 updates. The first update changes the wysiwyg_font_units option and empties the template cache. The second changes the enable_previews option and empties the template cache.
Each update should contain all the code needed to move a site along for the code being pulled for that update.

You may want a similar startup hook that customises database settings based upon domain name – for those small configuration differences between dev/staging/live websites (e.g. the hc_is_test_site setting).

Tips

Here are some assorted tips to help with portability:
  1. Avoid using Composr attachments because it's hard to sync them due to database changes. Instead place files in uploads/website_specific or uploads/filedump and reference them using the media Comcode tag (which has all the same rendering functionality attachments do).
  2. Avoid coding in the site's base URL directly, use {$BASE_URL}.

Helping users use the correct server

You can make favicon_staging and favicon_dev theme images. These (by default) override the favicon if these environments are detected. The development environment is detected by localhost. The staging environment is detected if the Health Check addon is configured with the "Test site" option enabled.

You can also make a Composr systems/startup hook to add a notice only for the staging site.

Code (PHP)

<?php /*

 Composr
 Copyright (c) Christopher Graham, 2004-2024

 See docs/LICENSE.md for full licensing information.
*/

/**
 * Hook class.
 */
class Hook_startup_staging_notice
{
    /**
     * Run startup code.
     */
    public function run()
    {
        if (running_script('index')) {
            if (strpos(get_base_url(), 'staging.example.com') !== false) {
                require_code('site');
                attach_message('This is the staging site, for testing out code changes. Content should be added to the live site and reviewed with Validation off, then on after approval.', 'notice');
            }
        }
    }
}
 

This code assumes the staging site is at staging.example.com.


See also


Feedback

Please rate this tutorial:

Have a suggestion? Report an issue on the tracker.