Composr Tutorial: The Composr programming framework

Composr is not just a web application, but also a programming framework. Composr defines a very large, fully documented, API, and has numerous mechanisms for the addition of code. Programmers are able to cleanly extend or override any aspect of the system, to enhance Composr or bend Composr in any area necessary.

It should be noted that Composr can be heavily customised without any programming, via editing the templates, but this will mostly only adjust layout and appearance, rather than adding-to or modifying Composr behaviour. Wherever possible we make things work on a configurable level though (such as Comcode pages, or Catalogues) so users with specific needs do not need to hire-in-help or learn programming to fulfil them for such cases.

It is also important to note, that this is a tutorial for programmers, and that programming tasks are inherently technical, and requires some combination of some of these elements:

• education
• experience
• a technical mind

This tutorial covers day-to-day aspects of making and installing addons, especially embedding or overriding code. For a more exhaustive document describing ground-up Composr programming, see the Composr Code Book.

Open Source

Composr is Free and Open Source Software, meaning it's completely open and customisable.

The Composr API

Composr has a well documented API (code)

(Click to enlarge)

Composr has a well documented API (compiled out)

(Click to enlarge)

Every function in Composr is documented using a system based on PHPDoc (itself based on 'javadoc'). One purpose in this documentation is that it makes it easy for a programmer to program for Composr: they have a few hundred fully documented Composr functions available to them.

With PHPDoc and the function header-line itself, every function has the following:

• A description
• A list of all parameters:
  • The code-name of the parameter
  • The type of the parameter (including whether false [~type] or null [?type] values are accepted)
  • A description of the parameter
  • Whether the parameter is optional
• The return type (if any), and a description of it

Of course, the actual function code is also available. All this information is presented as a part of the API guide. We also have the Code Book which includes a lot of additional information about programming for Composr, and is one of the actual sources of information for the staff; it includes a lot of important and expert information, including guidelines for writing secure code.

Extending Composr

This section provides a high-level overview of Composr's core programming. Much more thorough documentation is in the Code Book.

Composr is not just a lump of code like most software: a lot of effort has gone into structuring the system so that code may be cleanly hooked in, via various methods.

These methods include:

• Overriding of existing source files with modified versions. Every directory that holds effective source code (as opposed to the shell files such as index.php or dload.php that just provide entry points into Composr) support a system of overriding: in other words, every Composr source code file can be replaced (or extended) with a customised version, cleanly, without damaging the existing code and without the changes getting buried.
  • This is done primarily through saving such files in _custom directories with the same name as the original file; Composr will then elevate these files.
  • Contentious override hooks can be used for more sophisticated overrides (such as replacing something in the code, injecting something before or after a specific line, or introducing logic to decide when to apply an override). These are located in sources_custom/hooks/systems/contentious_overrides.
  • If code gets changed because of an override, Composr will generate a compiled version under _compiled for requiring into PHP. This ensures developers can still make use of stack traces and that PHP's native cache extensions (when active) are still utilised for the overridden code.
• A Composr editor (the code editor) is provided to provide a simple (password protected) editor for your site, and automatically saves your changes as _custom overrides
• New Modules can be added just by writing a file and placing it in the modules_custom directory for the zone that it is to run in. In addition, there is support for a simple form of module, called a Mini-module that can be added in a similar way, but is much easier to write for amateur programmers
• New Blocks can be added just by writing a file and placing it in the sources_custom/blocks directory. In addition, there is support for a simple form of module, called a 'mini-block' that can be added in a similar way, but is much easier to write for amateur programmers
• New API source code files can be added just by writing a file and placing it into the sources_custom directory
• 'Hooks' to add features into a hooked area can be written. For example, a 'search hook' can be written by writing a file and placing it into the sources_custom/hooks/modules/search directory.

An example of a new module

Composr language files for the tester addon

(Click to enlarge)

Composr templates for the tester addon

(Click to enlarge)

As a part of testing ocPortal 2.5 (along time ago now), we wrote a new custom module that provided a quick way for all testers and developers to progress for the testing/bug-fixing process in a collaborative fashion. The module allowed the addition of tests, and assigning of those tests to members in a testing usergroup. It had a security system, so that non-testers could not access the system, and so that security could be defined to determine which testers had write access to edit tests. It also had forum integration, so that bug report topics could automatically be created and associated with a test.

All this was done, without touching the core Composr code. The Composr API made all the integration (security, members, usergroups, forum topics, etc) very easy. It is not the intent of this tutorial to explain exactly how this module was written, but some screen-shots have been attached to illustrate where new files were created in order for it to work.

The process of writing the module went along this basic route:

1. The functionality of the module was decided upon, and tests designed that would allow us to test all this functionality worked once the module was finished (we usually define tests first, write a module in one go, and then carry out all the tests: it usually is the most efficient and robust way for us to write the code)
2. The database schema was designed for the module, by consideration and then simply listing of the tables and fields that would be needed to adequately (and consistently with our standards and the guidance of our expertise)
3. The shell for the main module was written, based on that which all other modules (other than mini-modules and abstracted CRUD modules) uses
4. It was decided what screens that module would use, and these were all assigned code-names. Functions were created for these code-names, and the run function was filled in to call them up according to the type parameter (as featured in all other modules)
5. Install and uninstall functions were written for the module, according to the database schema
6. Add, edit, and delete backend functions were written for the data of the module
7. Add, edit, and delete screens were written for the data of the module (including definition of associated language strings for the standard Composr field inputting interface [wherever possible, referencing existing ones, to reduce the burden for anyone wishing to translating the module once we release it publicly as a Composr addon])
8. The screens for the main interface were written
9. The module was tested, and then deployed

Composr module for the tester addon

(Click to enlarge)

The new module, approximately 1,000 lines of code (PHP and XHTML) was completed within one long day, and in active use soon after. It is a moderately advanced module, and was of great help to our organisation. Naturally, it would take a lot longer for an inexperienced Composr programmer to create this module, but the point is that it was made a lot faster due to our API and extension infrastructure: without these, a custom application from scratch might have taken a full week to develop. The alternative to a custom module would have been a third party application, but this would not have provided us with the level of integration we demand for our systems (not to mention compatibility with things as they are currently, and how they might change in the future). Professional developers are available for implementing systems built around on Composr (as third party developers may be also).

Some of the API functions used in the module include:

• create_table
• add_privilege
• query_select
• do_template
• do_lang
• has_privilege
• build_url
• assign_refresh
• get_param_integer
• get_member
• member_group_query

I'm sure you can see why such a function library (and the pre-written subsystems that exist behind them) speeds up development considerably. All this is built into Composr by default, so no libraries need hunting down, installing and managing to achieve all this.

Overrides

Composr lets you override individual PHP files with changes by adding to the sources_custom directory, which parallels the sources directory. For more fine-grained control, you can also make a contentious_overrides hook. Overrides take precedence.

Overrides work at the file level. When Composr is running code it is rarely doing anything at an addon level, addon management is more to do with initial packaging/install, and then it's just files.

If you are overriding a file you are taking responsibility for merging in any changes made to the original file back into your version (the upgrader file integrity check can help).
If some other non-bundled addon is overriding the same file, it is up to you to resolve the conflict by doing one of the following (most preferred method at the top):

• Creating a new set of hooks for the conflicting functionality (which also allows future developers to expand on it)
• Creating a contentious_overrides hook for programmatic alteration
• Merging the overrides together (not recommended because ideally, each file should only belong to one addon, and Composr only supports this)

It isn't feasible right now for us to create any mechanism for multiple people overriding the same files, as this would just create a complicated tangle in the long-run. However, the issue tracker may have an issue suggesting rearchitecturing of Composr for the future to support this.

There are 4 different kinds of PHP override in Composr:

1. Class extension – a special case where we allow automatic instantiation of a derived version of the original class
2. Replacing/supplementing a file, with your override loading first, then any non-overridden functions/classes from the original loading second
3. Replacing/supplementing a file, with the original loading first, then your override
4. Replacing/supplementing a file, with support for programmatic alteration of the original code – actually this can be done in tandem with techniques 1 & 2 above, but not so easily technique 3

It's important to note that as well as overrides you can place entirely new files in the *_custom directories, and Composr will be able to compile them as if they were original files. In fact, this is what non-bundled addons do. And ideally, you should do this as well for any new code you write. That way, changes are not lost during upgrades.

All these techniques are handled inside the Composr sources/global.php file. If you are a good programmer but still can't work out how to do all this stuff (it's easy once you know how, but a bit unorthodox compared to normal techniques), look at the code of this file.

Note that the following files cannot be overridden:

• sources/bootstrap.php (this is Composr's entry point into the API)
• sources/global.php (this contains the code override system itself, so you cannot override the override system)
• sources/minikernel.php (a slimmed down API which does not contain the robust code override logic that global.php does)
• sources/crypt_maintenance.php (could be called outside of Composr's API)
• sources/version.php (could be called outside of Composr's API)
• sources/file_permissions_check.php (technically can be overridden, but if fixperms is run without the –full parameter, the overrides will not apply)
• Anything that is not loaded in through Composr's require_code function (e.g., third party code)

Also be aware that overrides do not apply when Composr is running in minikernel, and often do not apply when running in safe mode. This is by design so that the core Composr system can function even if something else is breaking it.

Where possible it is good to use hooks to add in functionality instead of overrides. For example, you can write hooks that listen for particular database queries, then do additional things when those queries happen. As hooks are individual unique files, rather than overrides, there's no problem there. But it requires us predicting in advance where hooks would be useful, and implementing them, and there'd be an overhead to us allowing more of that.

If you are developing a non-bundled addon and feel the need for a type of hook that doesn't exist yet in Composr, please create an issue on the tracker.

Extending a class

This form of overriding works via class inheritance. All you need to do is to make a *_custom directory version of an original directory file, with a few things to keep in mind:
1) You need to use a special prefix:

• For Module_*, use Mx_* extends Module_*
• For Block_*, use Bx_* extends Block_*
• For Hook_*, use Hx_* extends Hook_*
• For Source_*, use Sx_* extends Source_*

1. Make sure the top of the *_custom file you created contains the /*FORCE_ORIGINAL_LOAD_FIRST*/ comment, otherwise you will get errors about the extended class not being found. This forces Composr to load the original file first, which ensures the original class is defined when you extend it.
2. As this works via standard class inheritance you are at liberty to, and should, only re-define methods in your override when those methods are actually different from the original ones. i.e. start with a shell of a class and only re-define methods that you are actually wanting to change. And remember that you can use parent:: when you want to call the original logic on that method.

Note that these classes cannot be overridden:

• Tempcode: This is treated as a data type in Composr
• CMSException: This is a special PHP exception defined by Composr

When you override a class (and define the comment /*FORCE_ORIGINAL_LOAD_FIRST*/), the following happens in the code compiler:

• The class name in the original file is renamed to the special "x" prefix.
• The class definition in the custom file is reversed (e.g., Mx_* extends Module_* becomes Module_* extends Mx_*).
• The compiled original file is loaded first, defining the "x" prefix class.
• The custom file is loaded second, defining a class with the original name which extends the "x" prefix class in the compiled original file.

This may be a lot to wrap your head around, but this process ensures the following:

• We do not need to modify every file which calls the overridden class so it calls a new class name
• Calls to static functions and properties on the original class name still work, and in fact, will inherit any overrides you define
• We do not need to put in any special considerations for whether a custom class exists; the code can continue calling the original class name

Replacing/supplementing a file

If you override a file to the sources_custom directory then that file will essentially be overlaid on top of its equivalent in the sources directory. (You can use this technique in other *_custom directories too, such as modules_custom directories – although in this case it is less useful, as there is less granularity due to a module only containing a single class)

You can redefine any existing function or class, and you can define new functions or classes. Composr does some magic so that you can reference the old versions of the functions by prepending non_overridden__ to their names (e.g. example_function would become non_overridden__example_function). This is very useful if you can write your overridden function without having to completely replace the original function. For example, if we just want to add some logging to example_function…

Be aware this does not work for classes. If you want to override a class, you must use a method defined in the "Extending a class" section above. You can, however, define new classes in your custom file without any special considerations.

A word of caution – if you are not supplementing a code file's init__* function, then you will want to strip out the init function from your override, otherwise both it and the original will be called in sequence. This is unavoidable, because the modified file is loaded first and thus cannot call the original init function itself – and thus responsibility for that stays with the Composr code. There are two exceptions:

1. If your override contains every class or function the original file has (i.e. you overrode the whole file by copy&pasting) then the original init function will not be called as Composr assumes you copy&pasted all that code into your own init function (we use the logic, "if it's 80-120% the size of the original, then it's probably a copy and paste.").
2. If you want a partial override but don't want the original init function to run, you need to use programmatic alteration to mask the original init function. This is explained in the last paragraph of the next section.

There is one important rule when it comes to init functions: you are not allowed to call non_overridden__init__<whatever> from your init__<whatever> function. You cannot do this because the PHP code containing that function has not been evaluated at that point and thus the function is not yet defined. If you try and do it then Composr will disable programmatic alteration and hence revert to the more basic 'Replacing/supplementing a file' technique. Composr will automatically call non_overridden__init__<whatever> as soon as it becomes available so you do not need to copy&paste its contents to your own function. If you do not want non_overridden__init__<whatever> to be automatically called then you need to rename it in your own init function so that Composr cannot find it.

A second sticking point is when you are extending a class defined in the original file. In such a case you need the original to load first. Do that by including /*FORCE_ORIGINAL_LOAD_FIRST*/ in your code. You will need to make sure that no function/class names conflict, which you can also do with "Programmatic alteration" (see next section).

Programmatic alteration

Sometimes you want to override something in the middle of a function and thus there is no neat way to do it without copy&pasting the whole function. Sometimes you also need to rename conflicting function names if you've defined the original to load before your override.

In theory, good software architecture (modularity etc) is meant to prevent this situation happening, but in practice:

• its completely impossible for an original programmer to predict what changes someone might want to make to their code
• programming languages have no inbuilt features to work around problems stemming from this lack of foresight (even the most heralded OOP techniques cannot do it)

There are 3 ways this problem could be solved:

1. Alter the original Composr code. This is a very bad idea because you then have a very hard time identifying your changes, and performing even patch upgrades
2. Override whole functions. This is a bad idea for non-trivial functions because it makes feature/major upgrades much more problematic
3. Use a mechanism that goes beyond what programming languages can normally do

We took approach '3', and thus have written a special feature into Composr for it. Its admittedly a bit messy but in practical terms it is a very powerful tool.

In Composr it works as thus:

1. First, you create a contentious_overrides hook in sources_custom/hooks/systems/contentious_overrides. Generally, name it the same name as your addon, but you do not have to do that.
2. Within the Hook_contentious_overrides_file_name_without_extension class, define a public function compile_included_code. See Code Book, part 1b (Further back-end concepts) under the section "Contentious override hooks" for an example to see how to structure it.
3. Put your addon guards in at the top of the function which return nothing if not installed.
4. Put your $codename guards in place, doing different things depending on what code is being included. You want to make sure your overrides only apply to the applicable files you want to override and nothing else.
5. When you know for sure you are about to modify the code, make sure to populate $code with the file contents of $path if it is null.
6. Finally, you can define your overrides. You can use any string functions like preg_replace or strpos, but it is highly recommended to instead use the handy Composr get_function_hash / insert_code_before__by_linenum / insert_code_after__by_linenum / insert_code_before__by_command / insert_code_after__by_command / remove_code functions from the override_api source. This is more elegant and will also output proper errors should the overrides no-longer work.

Exporting addons

The Code Editor

PHP programming help

It is beyond the scope of the Composr documentation to explain how to program in PHP (although we do have a stab at it in the Introduction to programming tutorial), or other languages that are used such as SQL or HTML. PHP includes an excellent reference guide, and there are many good tutorials on the web for HTML. Composr uses a very minimalistic form of SQL, and usually code does not need to use any directly, due to our database abstraction functions, so this should not be a problem.

The Composr API guide does actually include a PHP reference that defines a subset of PHP that we allow ourselves to use in Composr. The subset is specially limited so as to avoid PHP version conflicts, and the need for PHP extensions that may not be installed.

Other advantages to our framework

If you still are not convinced that Composr is the right choice for you, consider some of these:

• There are many frameworks for creating web applications available (such as Typo3 or Ruby-On-Rails), but few true frameworks like Composr are also 'out of the box' systems.
• In addition to the above, few pure frameworks provide the dimensions of functionality that Composr can provide.
• Composr's framework has excellent security. For example,
  • the database abstraction system allows relational databases to be accessed without concern of SQL-injection;
  • The template system and JavaScript framework are written to make XSS injection virtually impossible (where the vast majority of programmers write code that is full of XSS vulnerabilities, without even knowing what they are).
• Composr's framework is of professional quality. If you opt for frameworks which are incomplete, and end up using addon libraries to achieve additional functionality, you will soon realise, both immediately and after-time, that the lack of central control results in:
  1. major compatibility problems;
  2. messy feature overlap;
  3. inconsistent philosophies for code;
  4. inconsistent philosophies for documentation;
  5. a large number of agents to contact for different kinds of problem;
  6. no central authority for you to agree licensing with, should you need to do so (and this is not unlikely, as many projects flower to unexpected ends);
  7. orphaned projects that die

See also

• PHP documentation
• MySQL documentation
• Introduction to programming
• API guide
• Code Book, contents and introduction
• PHP website

Feedback

Have a suggestion? Report an issue on the tracker.