#799 - Automatic automatic theme upgrades

Identifier #799
Issue type Feature request or suggestion
Title Automatic automatic theme upgrades
Status Closed (duplicate)
Handling member Chris Graham
Addon General / Uncategorised
Description Introduction
============

We want to be able to help users automatically maintain their theme compatibility, even if we make changes to the default theme (i.e. base/reference theme) between releases. Our changes may be small (e.g. small layout fixes in a patch release), or larger in a major release. Many classes of theme incompatibilities/issues can occur, such as:
a) interface changes between Composr and the theme files
b) our wish to propagate fixes into user themes
c) compatibility between different combinations and versions of inherited and overridden files (e.g. a CSS file's class names matching the HTML templates)

Automation is key in our ecosystem. Nobody is going to have time to manually create patch scripts between each version.

We don't want to have to be reluctant to improve our theme with new versions, our of fear of causing problems to users.

Patch generation
================

Diffs can be highly fragile - they break very easily for small changes of context, and are not end-user friendly. We can generate patch operations that lock on to narrower (but known as valid/unique) contexts. We can also take advantage of known document structure (i.e. CSS) to add additional robustness when byte-by-byte precision is not needed.

We can use git within our process as a stepping stone to making our patches, leveraging that git actually does some of what we need quite well already. We initialise a git repository with the old version's files, and then replace it all with the new version's files, and ask git for a "status" and a "diff". The "status" tells us things like filenames; the "diff" is a basis for creating our patch operations.

Our patch operations should be something like...

ADD_TO_SELECTOR (.css only)
DELETE_FROM_SELECTOR (.css only)
ADD_FILE <filepath, data>
RENAME_FILE <filepath, newfilepath>
DELETE_FILE <filepath>
STRING_REPLACE <filepath, beforetext, aftertext>
STRING_REPLACE_AFTER <filepath, precontext, beforetext, aftertext>
STRING_REPLACE_BEFORE <filepath, postcontext, beforetext, aftertext>
STRING_REPLACE_BETWEEN <filepath, precontext, postcontext, beforetext, aftertext>
STRING_REMOVE <filepath, text>
STRING_REMOVE_AFTER <filepath, precontext, text>
STRING_REMOVE_BEFORE <filepath, postcontext, text>
STRING_REMOVE_BETWEEN <filepath, precontext, postcontext, text>
STRING_MOVE <filepath, oldprecontext, oldpostcontext, text, newprecontext, newpostcontext>
ADD_AFTER <filepath, precontext, text>
ADD_BEFORE <filepath, postcontext, text>
APPEND_TO_FILE <filepath, text>
PREPEND_TO_FILE <filepath, text>
INDENT <filepath, text, newtablevel>
INDENT_BETWEEN <filepath, precontext, postcontext, newtablevel>

A context is simply the shortest unique character sequence we can find that precedes/proceeds where we need to perform our operation.

There will be situations where an operations parameters are massive. As far as possible we want to try and be smart, and detect things like white-space differences (e.g. indentations), to minimise them. We should generate test cases to do as good a job as we can.

We can store our operations in a PHP file, just as function calls (each op is a function). We can store the file in data/theme_patches/<version>.php). The assumption is that <version> upgrades from whatever was the previous version to it.

Generation of patches should be integrated into our build process, as an intermediary step between inputting generation settings and generating the build archives. It should be able to automatically download the latest manual package from compo.sr, extract it, do the git stuff, then build the patch PHP file. The patch file should automatically be "git add"'d to the Composr repository being built. The human operator should then be informed that they can customise the patch file, as undoubtedly it would be better tuned (e.g. removing unnecessary ops).

Raw diffs
=========

It would not hurt to also save the raw diffs and git status reports into git. These could be manually generated, and aren't needed by our process, but users may find them useful if we package them up automatically.

Patch application
=================

The upgrader (sources/upgrade.php) already has some code for performing patch operations, but they are hard-coded in PHP. This will need replacing with the implementation of the patch operation functions, and general code to go through each patch between the version being upgraded from and the version currently running.

When Composr performs operations it should be smart enough to pipe through the Theme Wizard, as needed. New text/data that is CSS or a theme image may be altered by the Wizard. We also need to create revision files automatically (file.ext.timestamp).

Humanisation
============

A second patch reading tool should be developed. This turns the patch ops into something intended for humans to read, by outputting Comcode instructions, making heavy use of the Composr 'code' tag.

For example,
{STRING_REPLACE foo.css, acronym, abbr>}
could become,
"
[title="2"]foo.css[/title]

Change all 3 occurrences of acronym to abbr.
".

Two observations:
1) We are grouping operations by file.
2) We are grouping identical operations. In this example, the op was happening 3 times, so we put in one human instruction, with the qualifier "all 3".

We can build this feature into the upgrader also, as a link to view manual changes between any two versions.

We can further extend this by adding annotation support to our PHP patch files, as a further optional parameter to each op. These are comments intended only for human reading, where we explain why each change has happened, or any other particular notes we want to attach.

Feedback
========

Even better if we can combine the automated upgrading, and the human instructions. When the automatic upgrading has happened, show all the human instructions with a checkmark or a cross next to each. This lets the user clearly see what steps succeeded and what failed.

Automatically save these results into a dated HTML log file. In fact, start saving all the upgrader operations into log files, and show these in the upgrader as a history list.

Notes
=====

This process is inherently a number of steps. We can develop each step independently in its own code module (sources file), to break down the implementation in a more manageable way.

Note the existence of PHP's levenshtein function. I am not aware of how this may be useful to the algorithms here, but it is the kind of thing that might be.
Steps to reproduce

Funded? No
The system will post a comment when this issue is modified (e.g., status changes). To be notified of this, click "Enable comment notifications".

Rating

Unrated