Composr Tutorial: Performing an upgrade

This tutorial will give background information for this process of upgrading Composr. The Composr upgrader does step you through the process, but further information will be given here so that you may gain a further understanding.

Types of upgrade

When the developers make a release they will state whether they recommend users upgrade to it. If they have corrected a security issue, or a major bug (such as one that causes data loss), they are likely to strongly recommend upgrade. You can see the fixes included in a release by looking at the bugs database for the release before it.

Upgrades may be roughly categorised under:

• patch releases – these are likely to involve bug fixes, with very few, if any new features. Any new features would have been implemented solely to provide important functionality that was obviously missing. Patch releases are very easy to upgrade to
• or, feature releases – these are likely to include new features; the quantity of new features will vary. Minor releases are usually easy to upgrade to but may have a few things to consider
• or, major releases – these are likely to involve many architectural changes, as well as new features; major releases are less common, as they need to perform additional testing for them which takes a long length of time – they therefore try to avoid architectural changes other than in large batches

Some words of caution

The developers cannot make any guarantees about the speed of which any new bugs might be patched, for any upgrade. It is your own responsibility to make and test a full website backup (files and database) before you upgrade, for use in the unlikely event of you wanting to revert your upgrade.
Some users may wish to test upgrades (and pre-fix any theme compatibility problems that could occur) on a staging/clone site (a private mirror of their live site).

Special note: multi-site-networks

If you are using Conversr on a M.S.N. and upgrading to a new major release then you will need to upgrade all the sites together, starting with the one that hosts the forums.

Special note: upgrading third-party forums

If you are using a third-party forum and upgrading that software, be sure to check that the Composr forum driver you are using is compatible with the new version. This is usually only an issue with major new releases of third-party software. If you need a new/upgraded forum driver, professional developers are available for such work.

Things to check in feature/major releases

Be aware that a number of things can change significantly across releases. Make sure you have a good understanding of how the following things may affect you, before making 'the jump':

• Template and CSS structure could change considerably. Files may be renamed, reorganised internally, or things may just be completely rearchitected.
• Occasionally features may be deprecated.
• New features will be added, and that could possibly interfere with highly tuned layouts or site flows.
• Minimum requirements may have changed, for example a newer version of PHP may be required as a baseline.
• Non-bundled addons may no longer function.

At the same time, bear in mind that fixes to security holes are not always back-ported to older releases, especially if a newer feature/major version has been out for a while. So ideally you should not stick around on old versions too long. If you want to avoid upgrading but also need to keep your site secure, you should strongly consider getting a developers support so that you can be kept up-to-date manually.

The upgrade process

The upgrader

(Click to enlarge)

Logging into the upgrader

(Click to enlarge)

The upgrade process involves a number of stages (the upgrader will step you through this)…

Pre-steps:

1. get to the upgrader, either via the upgrade link from the top-right of the Admin Zone dashboard (which may take up to 3 hours to update due to cache), or going to http://yourbaseurl/upgrader.php
2. log in to the upgrader using your maintenance password (the one you set when you installed, different to your admin login)

Steps:

1. taking a full backup (just in case a problem occurs) – better safe than sorry. This is also useful in case you overwrite files that you have edited: either due to these files being in locations with no override support, or due to changes being made without using the proper override mechanism
2. closing your website
   
   • the upgrader will make it easy for you to close your site using the Composr closed-site feature
   • however, for non-patch upgrades your website may return errors until the full upgrade process is complete, so you may wish to place your own closed.html file on the server that shows a standalone closed-site screen. Composr will redirect to that file for all page requests for as long as it exists on your server (so it does not need to load its database or templating system)
3. transferring new/updated files (this will copy your customised upgrade file between https://composr.app and your server, and then extract it)
   
   • if your server's connection to the https://composr.app server is slow, you could download the TAR file yourself, upload it to your own server, and then use that copy's URL in step 3
   • on modern server environments the extraction happens in a separated framed process, so that if timeouts occur it can't leave your website in an inconsistent state
   • if in the unlikely event that you do end up with an inconsistent state then you can download the TAR file yourself, extract it manually, and then continue; this is not ideal but the file integrity check should help remove files from uninstalled bundled addons that don't belong
4. resolve file integrity issues
   
   • if you have made any Composr overrides in place you may need to change these overrides to be consistent with Composr's latest structure; or if the overrides came from addons, you may need to overwrite the addon(s) with a newer version
   • the above guidance applies to totally new code also, such as new modules from non-bundled addons
   • resolving any conflicts with templates, as necessary (for example, if there was a change in the verbatim version of a template, and the upgraded website had also changed that template, then changes might [depending on importance] need to be incorporated manually)
   • removing any files from previous versions of Composr that are no longer required
   • removing any files relating to bundled addons that you have previously uninstalled
5. upgrading your database; this is the most key step, which upgrades all addons, modules, and blocks, as well as doing some cross-system updates, and updating Conversr
   
   • as upgrades can be very resource-intensive (especially for major upgrades which may call for restructuring of large amounts of site data), the database upgrade process runs one addon, module, and block at a time, to help reduce the risk of server time-outs (but at the cost of the database upgrader taking longer to complete). The database upgrader will tell you when it is done.
   • if any errors happen they'll be displayed for a short time (and also logged to your error log so you can refer back to them again), but Composr will generally try and continue anyway if there are minor errors happening.
   • Database errors are common on the last "bleeding-edge" step because this often contains SQL queries which may have already been run by the upgrade of an addon, module, or block. They can usually be ignored on this step. This step is there if we had to modify the upgrade code in an addon/module/block (without bumping its version number), and therefore some sites already on that module/block/addon version won't normally get the upgrade.
6. performing an automatic theme upgrade (maintenance status) (if available for the version jump you're doing – we usually have this disabled because it is not something we can support well currently; instead we recommend using https://composr.app/themeing-changes.htm to help you do it manually, and also using the "Theme Wizard files" Website cleanup tool)
7. Running another file integrity scan (for minor/major upgrades) to be extra sure all issues were addressed
8. If you use an .htaccess file, comparing it with recommended.htaccess (which may have changed) and merging in changes
9. clearing caches
10. re-opening your website

Browser cache

Often browsers will cache CSS and JavaScript files, even when changed on the server. Usually completely restarting the browser will make it recheck cache times. If you get JavaScript errors after upgrading, chances are fair that you need to empty your browser cache. If your site runs on a CDN, such as Cloudflare, you may also need to purge the cache there.

Assistance

Upgrading is inherently complex, as it is close to programming in the sense that low-level changes are being made to the structure of the website. This is unfortunate but inevitable, and due to Composr's openly configurable and overridable (at the code level) nature, the developers would have an extremely hard time trying to automate the process beyond what the upgrade system (the stage launched by running http://yourbaseurl/upgrader.php) already does.

If you require assistance for an upgrade, professional developers may be able to help.

The integrity checker

Results

(Click to enlarge)

Finding the integrity checker

(Click to enlarge)

This section contains standalone details on using the integrity checker.

Composr comes with a built-in "integrity checker" which can be used to repair a corrupted copy of Composr. Integrity checker is a part of the upgrader tool. The integrity checker can be used to find outdated and corrupt files in the Composr system. You can use this tool at any point, regardless of whether you have recently upgraded or not.

You must provide the maintenance password to access the Composr upgrade tool.

The integrity checker is placed in the "Step 4" of the table under the "Upgrade steps" section in upgrader tools page.

The integrity checker will display the following kinds of file (if any):

• Files that are actually missing and need uploading:
  • Missing the original, no override ("These files are actually missing and need uploading")
  • Missing the original, has an override ("The following original files to these overridden files are actually missing")
• Overrides blocking original files ("The following files have been superseded by new versions")
• Modified files:
  • Newer than the original ("These files do not match the ones bundled with your version, but claim to be newer")
    • These can happen if you apply hotfixes from the tracker
  • Older than the original ("These files are outdated")
  • Older than the original, and with an override blocking the original ("The following non-overridden files are outdated")
• Files that maybe shouldn't be there:
  • Alien files (unrecognised files)
  • Files from non-installed addons
  • Files that have since been moved

If you want to prove to yourself how it works, create a random php file named somerandomname.php in the sources directory of Composr and run the integrity checker again. Now it will list the new somerandomname.php file in the results (because this file does not belong to any addon).

This is very useful for upgrading because it helps identify old files that need removing. Generally though it is also useful to make sure unexpected files don't lay around within the system causing potential problems.

The integrity checker has the ability to get rid of the alien files by quarantining them (moving them under an _old directory). Check (tick) the checkbox next to the file name and press the "Automatically handle those ticked (checked) – quarantine/move as needed" button placed below the list of files.

Obviously you should be careful quarantining files, as there may be legitimate reasons you have put alien files there (e.g. domain ownership validation files, Project Honey Pot integration scripts, custom code). The upgrader is smart enough to recognise files from old versions and automatically tick (check) those, leaving the ones totally unrecognised unticked (unchecked).

Upgrade packages

Generally you upgrade using the package generated from the news announcement of the new version. A custom package for the version you are upgrading from will be generated. It is not safe to extract these packages manually because they contain the files for all addons, regardless of what you still have installed or not. You do not want to get files from non-installed addons back into your system, it can cause many issues. When you install the package from within the upgrader it is smart enough to know what to extract, verses what to package into the addon TARs that were archived to disk when you uninstalled those addons.

In the file transfer step of the upgrader you can see there is also a link to generate a custom upgrader specifically for your site. This link is more customised than the above because the package is generated with knowledge of what bundled addons you have installed, and thus it is possible to manually extract it safely. The package also contains a full set of non-bundled addon TARs so that you know anything you reinstall will be for the version you are currently running, not an older version.

We may very rarely provide something called an "omni-upgrader", which can be used to upgrade from any version. We did this when we switched from ocPortal to Composr (v9 to v10), as our entire infrastructure got cycled over, breaking the normal upgrade workflow. For v10 to v11, however, an omni-upgrader is not provided because v10 sites (from 10.0.48 beta onwards) are able to upgrade to v11 as normal through the upgrader.

Auxiliary features

There are some extra features in the upgrader that aren't a part of the normal flow:

• Checking/fixing necessary/excessive file permissions
• Delete all files from a specific addon
• Opening up the site in Safe Mode
• Correcting MySQL schema issues (this is intended for programmers-only, you are very unlikely to need it)
• Fixing MySQL tables (this will try and fix some database tables; if the database has become very corrupt, you may need to use a tool such as phpMyAdmin to repair a database from outside Composr)
• Clearing caches (okay, this is a part of the normal flow, but it is very useful even if not doing an upgrade because you don't need a working Admin Zone to do it)

Troubleshooting

Timeouts

If the upgrader times out trying to download the upgrade package then try:

1. manually downloading it to your machine
2. uploading it to your server (such as in exports)
3. substituting the auto-generated URL with the local relative path to the file (e.g. /exports/upgrade_file.cms)

Other transfer/extraction issues

In the "Transfer across new/updated files" step there's a "generate a personalised manual upgrader" link. This will create an upgrade TAR file that is safe to manually extract onto your server.

Rescuing broken sites

You should keep Composr updated before PHP is upgraded, to maintain compatibility with newer versions of PHP.
If you fail to do this and find you can't even run the upgrader, or otherwise have a severely broken website without a working upgrader, this simple project may help you:
Composr ecosystem / Make Composr Upgrader · GitLab

See also

• Professional upgrading
• Cloning your site
• Backing up your site
• Problem and feedback reports, and development policies
• Theme Lifecycle
• Website Health

Feedback

Have a suggestion? Report an issue on the tracker.