Code Book, part 3 (Miscellany)

« Return to Code Book table of contents

Composr core performance

The Composr core is extremely optimised.

Most of the Composr page generation time can be put down to detecting the context the user is operating in, retrieving all components from their storage locations, structuring, processing whatever aspects of the output are dynamic, and outputting. This typically all-together takes a few hundred milliseconds. With opcode caching and persistent caching, it can be under 100ms.

The very core Composr responsibilities are as follows…

Input:

• Set correct character sets, for unexpected inputs (fast default case: ASCII request)
• Input sanitisation / checking / gating to defaults
• Input state management
• File/HTTP I/O stubs

Output:

• Find character set (fast default case: hard-coded character set optimisation)
• CSS and JavaScript inclusion/merging
• Breadcrumbs (disableable)
• Comcode-page display (fast default case: Comcode cache)
• Date/time display
• Time period display
• Output framing
• Tempcode assembly
• Tempcode rendering
• Tempcode symbol processing
• Tempcode output escaping
• Theme images (fast default case: persistent cache)
• Output state management
• Primitive type displaying
• Locales
• Basic template bindings

Internal dispatch:

• Finding linked scripts (internal caching)
• Error generation
• General link generation
• URL-moniker linking
• Page-link decoding
• Keep-parameter propagation
• Page-identification and dispatch
• Page searching (fast default case)
• Match-keys
• Hook searching
• Block and module calling
• Block processing
• Metadata querying
• Addon state detection (fast default case: persistent cache)

Configuration:

• Read base config and provide defaults
• Software versioning
• Config options API (fast default case: self-learning cache)
• Config values API

Caches:

• Find if a decaching request is active
• Static caching for Guests and bots (the ultimate optimisation)
• Persistent cache
• Self-learning cache
• Block caching

Context detection:

• Getting base URLs, based on SSL and zone context (internal caching)
• Detect safe mode (fast default case: hard-coded setting)
• Find current script context
• Mobile detection
• Rendering option detection [e.g. wide mode]
• Cookie detection (may be disabled)
• JavaScript detection (may be disabled)
• Browser capability/flag detection
• IP banning (disableable)
• Detection of user language (fast default case if internationalisation disabled)
• Self-linking/context-identification
• Detect of timezone (fast default case if internationalisation disabled)
• Server/HTTP-environment querying
• Users-online limit (disableable)

Users/Members:

• Find the current member
• Session restoration (fast default case: persistent cache)
• Session clean up (only runs randomly)
• Cookie logins
• Alternative login types
• Display names
• Custom Profile Fields
• Forum driver object initialisation and driver binding and driver
• Detecting and actioning logins

Processing:

• Memory management
• Intelligent code inclusion, overrides (fast default case: persistent caching of current code profiles)
• Database API (delayed connection instantiation; structural management code not loaded by default)
• Database API low-level driver
• Error handlers (only called exceptionally)
• Comcode (fast default case: simple conversions)
• Language requiring (fast default case: self-learning cache)
• Lookup language strings, support parameters, pluralisation differences, vowel/consonant differences
• Lookup translations, including linking to Comcode parsing
• Translation database storage
• Stats logging (disableable)
• utf-8 support
• Sorting
• Primitive types data conversions
• IP address detection and processing

We try and keep the core code size to an absolute minimum, due to memory requirements and PHP parsing time. Sometimes we have a little more code loaded up than needed, but only for things that are common even if not 100% routine. It is reasonable to assume a performance-conscience user will have an opcode cache, which removes the issue of code quantity.

Anything not marked with a specific bypass optimisation (in parentheses) will still have been heavily optimised: the Composr core is extremely optimal.

Note that while we have a lot of user-specific and security features in the above list, but these can be bypassed when not needed via the static cache.

Various hidden optimisation flags are described later in the Code Book, and also in the Optimising Performance.

Philosophy

Engineering standards and trade-off

When coding for Composr, you should be careful to make sure all our design goals are maintained. Sometimes trade-offs have to be made between design goals, sometimes an option can remove the need for a trade-off, and sometimes an option would just bloat and confuse the product. Good balance and thoughtfulness is required before rushing to add the latest cool ideas.

We have established the following design goals (in no particular order):

• Security – Composr should be completely secure and un-hackable, but also be able to track offenders and be easily restorable if the worst happens
• Performance – Composr should be as efficient as possible, with both CPU and memory usage; things like caches and optimised code, and sensible feature design, can achieve this. Tip: use xdebug for profiling, along with WinCacheGrind.
• Great design – Composr should look excellent on every page
• General quality – Composr should not contain things such as spelling errors
• Flexibility / generality – Composr should be able to work in many different kinds of website. Features should be usable in different ways for different people. It should not have a restricted layout (hence Comcode, modules, blocks, and templates)
• International – Composr should work well across the world, across language and culture (as long as someone puts the effort in to do a translation)
• Compatibility – Composr should work well with common versions of required technologies
• Modability – Composr should be modular, and overridable; easy to change without tearing the things to pieces, preferably without even editing things (just adding)
• Usability – Composr should be very easy to use. The best user-experience and interaction-design methods should be used.
• Power / features – Composr should be as powerful as any competing product, but that should not be an excuse for implementing crap that bloats and over-complicates the product; we all want loads of features, but we don't want to confuse the users or tie ourselves up so tight that future development is very difficult
• Standards compliance – An accessible web, based on open standards, which is stable and not tied to any specific vendor, is desirable for all; by sticking to standards, you are ensuring Composr works on a wide range of technologies, and for a wide audience
• Documentation – Remember that the best documentation is when the interface describes itself, but the written documentation must be excellent.
• Consistency – The whole system should have common ways of doing things. Everything should seem consistent to the user, and be so behind the scenes. Consistency is piece of mind, consistency is ease of use, consistency is simplicity, consistency is less code to test and fix. I can't stress how important it is to be consistent.
• Independence – Composr should not have unnecessary dependencies, such as MySQL.
• Stability / Error-trapping / Input-sanitisation – Composr should catch all errors, not let things like PHP notices slip through. This means that programmers have to get used to a new level of strictness, we are much less tolerant of errors slipping through because for complex software like Composr we can't afford weak foundations.
• Scalability – Composr must work on sites both small and huge. For example, it's wrong to put all downloads in a list on any single screen – there could be 1000's.
• Portability – People should be able to move their websites between servers without much difficulty.
• Simplicity – Features only needed by a minority should be "off-by-default"

Why not more object-orientation?

Most PHP CMSs have gone in the direction of having very sophisticated object systems, with dependency injection, design patterns and namespaces.

Composr however doesn't use this kind of stuff, and breaks what some people would consider best practices.

This is entirely intentional and we want to explain why, because there are a lot of people out there with strong opinions on how to design a web system and we realise our approach is not the conventional one that a lot of people are pushing. We want to explain why we think we have built a far more productive environment in Composr. This will seem like a bit of a wild rant, but really we've gone into detail to hopefully shine a light on the Composr design decisions.

It's a matter of good engineering to consider all factors when coming up with a design. Here are some design considerations, in rough order of decreasing importance:

1. Ease and efficiency of coding for the framework
2. Performance of live code
3. Ease of understanding and maintaining the framework
4. Ability to automatically test
5. Strict separation of concerns to stop developers interfacing incorrectly with functionality
6. Ability to create complex abstractions and contextual shifts when dealing with special scenarios – for example, test runs and simulations

'1' is so important. We feel that with OOP-heavy frameworks, where you have to worry about relatively complex initialisations of every part of the system you want to touch, and how you are going to plumb all the different components together, it really makes the code take a lot longer to write due to the extra verbosity. In fact, 66% of code can easily be taken up just by extra plumbing, and this really hurts when you're trying to get ROI on your time investment. Yes, extra plumbing is not that hard, but it is time consuming to do and maintain and to read through.

Regarding '2', method calls are expensive in PHP, and variable storage uses a lot more memory than you think, as does the memory simply to load up PHP code. So, there really is a high cost in all the plumbing you might want to do – 66% more code could certainly mean 40% more memory usage, and 30% slower CPU performance. Users increasingly expect sites to respond extremely fast, yet CPU speeds have not really increased in a decade. On top of all this, frameworks also mean you need to load a lot more code than you need, due to having to load up structures that are more complete and complex than you strictly need to serve the simple requests you are normally serving. Most OOP-heavy framework users respond by slapping on a full-page cache, but this is very limiting because then you really block the social interactivity you really want from a system like Composr.

Regarding '3', OOP-heavy architectures try to abstract away details to make things simpler, but in the process of defining custom interfaces for all modes of interaction, and plumbing to connect all the objects together, you end up with a huge amount of new interface complexity to both understand and maintain. We think this really is a zero-sum game. There's a really important design principle in the field of agile programming called 'You don't need it' (YDNI), and OOP-heavy architectures often go against this by trying to plan out for all access scenarios upfront and abstract them perfectly – it's much better that we don't waste resources (both programmer time, and system resources) until we need to make a particular access pattern work, and then we can optimise it for how we need it. Apart from cost/schedule, YDNI is also important because you can't predict future requirements well – trying to anticipate too much by putting in too much structure between components actually works against you rather than for you, giving you into patterns you actually would not have wanted. In other words, too much structure costs a lot to make and can tie you down as much as it can guide you.

'4' is definitely something OOP-heavy coding helps with. However, you can achieve equivalent things in simpler designs also.

'5' is another win for OOP-heavy systems, but it does kind of assume you are employing code monkeys, and that is never going to work however much you try and mollycoddle them.

'6' is something OOP-heavy systems do very well, but read on.

As you can see, simpler designs solve the most important concerns best, and then remaining concerns are well served by OOP-heavy designs. However, our Composr design is not actually a naive simple design at all – it's actually very carefully structured, and has a high degree of strictness/discipline in many areas. Composr is definitely not anti-OOP by any means, it is only anti-over-engineering – we actually rely on the strengths of OOP in many key areas of the system. Here's how our approach works to achieve the commonly-cited OOP-heavy advantages (flexibility, maintainability, clarity, and stability) without slowing development, complicating code, or hurting performance:

• We can achieve the equivalence to dependency injection as our output and execution context states support a stack-based switching mechanism, allowing us to create sandboxes very effectively for those few cases where it is useful. The critical difference is we achieve it in far less code, with better performance, in a way that is easier to understand, and where the burden for it is only felt when you actually need this advanced functionality, rather every time you use any component of the system.
• We can achieve inheritance via our code overrides system, which also provides the ability for users to override components without having to get down and dirty with coding. Different versions of code can be placed and tested as overrides, and new functionality can be placed using overrides. This does not suffer from the same problems as copy and paste coding, as overrides are designed to work on the function level, via some clever code we have in place, and we can even make line-by-line changes should we need to, or extend individual functions.
• We do have automated tests to test critical functionality. Admittedly these are more prone to break if we start redesigning stuff without confirming the interfaces used by the automated tests are still appropriate, but the difference is really not that much.
• We keep a very tight rein on global variables that we define. There is a formal naming scheme, and we have an automated test that scans the codebase and verifies that we don't start accessing any of them all over the place. Generally a global belongs in the file it is defined in, and from this point of view the source code files are like classes – the globals in them are their private properties. In essence, rather than enforcing this through verbose code plumbing and memory consumption, we automatically enforce it through automated testing for compliance to our coding standards. If a global is allowed for multiple access in different places (usually for performance), then we have a rule that it has to be given a formal annotation to what it does.
• We have the ability (via APIs and simple glue objects) to 'reflect' on content types within Composr in a really abstract way, even though the content types aren't defined by "fat objects". This provides the same power that rigid conformability allows, but without hurting performance or actually requiring rigid conformability.

The Composr design is actually based on how PHP is supposed to work, rather than Java-envy. Critics often think of PHP as inelegant, and it is true there are a lot of consistencies in the language and that some old legacy PHP features (that we don't use) were very poorly designed – but PHP is also widely popular because it is cost effective and easy to use, and this has been proven with top sites like Facebook using it. PHP is built on the concept that web requests should be simple and fast – they should get straight to the point and get what is needed done, and this is why PHP runs each request in its own 'sandbox' rather than as a thread on an application server. Web requests start up, do what they need to do, then are killed off. We therefore design Composr around the idea that these web requests are simple and fast, rather than trying to build a castle in the sky for a process that will terminate anyway as soon as its finished sending a result back. PHP environment access is based around 'super globals', just like Composr's access to key interfaces is – like the database connections. However, we improve super globals by adding the stack based context switching functionality described earlier.

So… hopefully you can now see past the anal arguments some people make that everything should be designed a certain way – an expensive, complex, way, that performs really poorly in the name of a theoretical but unrealisable maintainability. Don't believe any methodology's hype without doing a good analysis first – maintainability really comes down to the efforts the team does in keeping a codebase tidy and consistent, not much else. As good engineers we also know we want a framework that provides the most efficient possible coding method, in addition to the robustness, flexibility, and maintainability that we need. Knuckling down and over-engineering everything is going to make coding progress very slow/expensive and probably going to make code less maintainable rather than more maintainable – "simple by default" is a better plan if implemented with care and maintained with agility.

Rant over .

This is a good time to mention we have two frameworks derived from Composr called Conposr and Conposr++. These are built for standalone web apps and have a different set of design principles behind them.

Troubleshooting

There are a number of issues developers face when programming for Composr. Our high standards and development mode means issues are raised to the surface that less skilled programmers may not easily notice when working with other systems.

XHTML

Composr is coded against XHTML5, which is the XHTML version of HTML5. In other words, it is a stricter (in a good way) version of HTML5.

The extra strictness of XHTML is now largely historic, as no websites (including Composr) truly force the browser to enable the full XHTML-strictness, while browser XHTML modes aren't cut-down like they once were.
Never-the-less, it encourages good coding discipline – clean, intuitive code.

MySQL errors

Composr is not just built for MySQL. It is designed to work with any kind of SQL-based database software. It is all to common for developers to assume specifics of the MySQL platform are standard SQL and can be used freely. MySQL in fact is very non-standard in certain ways. Composr enables "MySQL strict mode" to force MySQL to (correctly) complain when mistakes (such as non-specified field values) are made.

It is advisable for you to:

• write SQL conservatively: avoid SQL syntax/functionality that Composr does not already use somewhere because that syntax/functionality probably is vendor-specific
• avoid being sloppy and getting MySQL to perform automatic type conversions and to automatically guess missing values during insert queries.

See the Using PostgreSQL with Composr tutorial for a practical discussion for PostgreSQL as a specific example.

Blank pages

If you find you are getting a blank page it is likely because you either have exceeded the PHP memory limit, or some kind of fatal error Composr/PHP cannot display.

Enabling the PHP display_errors and display_startup_errors in your main php.ini is a good idea (for a development server only) as it will allow you to errors that happen before Composr has started up.

Sometimes this advice will not help: if there is an "@" (error suppression operator) somewhere above the function call chain for where a fatal error happened then this will make fatal errors result in a silent blank screen. Composr cannot detect them. You should use "@" sparingly anyway.

Query limits

If you get errors about query limits consider optimising your code to use less queries. If you cannot do this, you can set:
…in your code.

You can also put &keep_query_limit=0 into the URL to have query limits temporarily disabled. Often when caching is disabled or empty the query limit can be exceeded.

Debugging, and stack traces

IDEs

Some IDEs have breakpoint support, watches, code-stepping, etc.

Simple but effective

When not regularly using IDEs, the core Composr staff usually debug via just putting temporary bits of code in, like:

You don't need all that code all the time, but if you don't use "exit" then you do need "ob_end_flush". This is because Composr uses an "output buffer" for its Tempcode system, and by default any output you do will get written into an arbitrary point in the Tempcode tree – often never to be seen at all!

In fact, rather than the ugly code above that just dumps information into the output stream, you can do something a lot fancier:

This should show the message in a nice position in the Composr output.

Of course, make sure you don't leave any temporary debugging code around when you finish!

Dev-mode

If development mode is running then the inspect function will be available in PHP. You can run commands like:
…and Composr will output the contents of these variables very neatly for you, and give a stack trace.

If output has already started then Composr will give you your details in an info box at the bottom of the displayed screen; otherwise you'll get a text dump.

This is very useful when stepping through code. Often bugs are because you are using the wrong data types (e.g. Tempcode objects being treated as strings). Temporarily placing inspect calls in your code and moving them along as you analyse exactly what is happening is a good way to test exactly what is happening for the occasions when the automatic stack traces are not available or do not reveal the real cause of the problem.

If you have an IDE with a debugger then this is better, but we find most developers do not have one, and also PHP debuggers tend to be quite unreliable. This old fashioned manual technique of manually tracing through code works well if you are quick at making changes and refreshing the browser window.

Stack traces

Composr will show a "stack trace" if it receives an error it treats fatally (a fatal error is one that terminates normal execution – because execution cannot finish). These stack traces are amazingly good at helping you find bugs, so if you get one have a close look at it.

A stack trace indicates the function call chain from the call of the script handling your current URL (at the bottom) to where Composr failed (near the top). In other words, as you look through the trace you are looking back through each function call all the way back to the top level script that the URL is calling. It doesn't show what the code has finished doing, only what it was doing when it failed and the function call chain that led to that action. At any level you can see the file the call was on, the line number, the function called from that line, and the arguments passed to that function (although large arguments like big arrays and Tempcode are omitted).

Lateral reasoning usually allows spotting of the source of a bug in a few seconds, although sometimes it can be more difficult. Often the bug is caused by unexpected parameters, and you can see them reflected in the stack trace and laterally work out why those parameters were passed and thus work out how to solve the problem.

One confusing thing about stack traces is that the error chain is included within the trace. With time you'll learn to read past this and see where the error was actually triggered. Typically anything above the fatal_exit or cms_error_handler function calls relates to the error handling process itself.

eval

If code is passing through eval (common with code overrides) then a stack trace can be difficult to analyse because you'll have a rough line number but no filename.

One technique is to turn on safe mode, because if the error still happens in safe mode you may also get a real line number.

Here's a helpful script to search a codebase for a function call happening on a particular line, to help narrow down a cause:

Tips

This section contains a number of general programming tips. Much of this applies to all programming, not just Composr programming. However there are also a number of tips for Composr that will help you on a day-to-day basis.

Setting default parameters

Sometimes on a complex project you will want a module to have a default Filtercode string, or a default virtual root, or some other default parameter different to what Composr hard-codes.

The best way to achieve this is to code up a startup hook. Using PHP code you can detect non-presence of a URL variable, and then set your own default directly into $_GET according to your own context.

For example, if you wanted a module as seen in one zone to have one virtual root, and the same module as seen in another zone, to have another – you could code up a startup hook that sets the default root accordingly.

Deployment

Be very careful deploying these particular files:

1. _config.php
2. .htaccess
3. .user.ini

_config.php is obvious, you don't want to overwrite the live server environment settings with development ones.
Sometimes it is a good idea to code up _config.php to detect what server it is on, then share that file across all platforms. This is mainly a personal choice.

.htaccess can be very server-specific, so be aware these pitfalls:

1. If a live server is CGI rather than an Apache module, over-assumptions can cause a 500 Internal Server Error
2. Any assumptions about Apache modules being there which may not be can cause a 500 Internal Server Error
3. If a RewriteBase line is there, it may be wrong for the server
4. The server could have its own file, with its own special settings (it's quite common to use the file to tune live settings)

.user.ini is similar to .htaccess in that it can encode very server-specific settings.

Tidying up ugly Comcode or running conversions

Often Composr users will write some horrible Comcode, that is impossible to reasonably extend without first cleaning it up.
Additionally, you may want to run some conversions, such as seeing what HTML some Comcode produces.

The data/comcode_convert.php can help you in these areas. It has options for converting between HTML/semihtml/Comcode, and has an optional reindent tool and HTML cleaner built-in.

Regular expressions

Have a good understanding of regular expressions and an eagerness to make use of them. You can often find uses for them, especially when searching code or preparing data-sets. They can save a lot of time manually reformatting things.

Webhosting

Never trust webhosts to provide decent hosting. Expect lots of bugs, unexpected limitations, performance bottlenecks, old software, and missing PHP extensions.
That's life, so try and minimise your dependencies and environment assumptions as much as possible – write simple, compatible, code.

git

It is in your interest to build up a decent understanding of Git, including:

• The very basics: Cloning, Adding files, Committing, Pushing, Pulling
• Creating branches
• Merging branches
• Reverting uncommitted code back to the last commit (git checkout -- <path>)
• SSH keys
• Resolving conflicts
• Creating diffs of uncommitted code
• Creating diffs between branches

Git is a useful tool for maintaining history on a project, deploying across servers bypassing issues of slow Internet connections (far better than FTPing), and testing experimental changes – even if you are working alone on a project.
It will save you a lot of time, and give you more security and stability in your work.

Writing new code

Many programmers struggle through things due to:

1. failing to split up tasks, getting overwhelmed
2. not writing clean code and confusing themselves and others (not splitting things up [abstracting] properly, copy & pasting too much, poor variable naming, poor code structure, poor code layout)
3. patching over problems via workarounds that "seem to work", rather than properly understanding what's going on

The solution is to be logical, calm, and rigorous. Also, over time your ability to build up a mental model of how things fit together in large code-bases like Composr, will improve.

Here is a good workflow for writing high-quality code for Composr:

1. Work out generally what needs doing.
2. Create a new feature branch in Git, if you are doing dangerous/experimental changes or are coding for core Composr (as opposed to a client website).
3. Lay out the basic structure of how the code will come together (files, classes, functions).
4. Document the solution using pseudo-code comments throughout the structure. Put a "TODO" comment under each of the pseudo-code comments.
5. Fill in the TODOs with real code. Make sure you are writing quality code and matching appropriate coding standards.
6. Run the Composr Code Quality checker on the code. Specifically: recalculate function signatures, then check the files you've added/changed. Fix any errors found, it should be possible to have zero reported errors with the default settings (all of Composr passes).
7. Write automated tests to prove to yourself it works well, if appropriate. This makes most sense if your code implements algorithms or APIs. User-interfacing testing is a lot harder, more subtle, and the tests more fragile, so frankly this is usually left to human testers.
8. Test manually / debug.
9. Add new files to Git, and Commit/push (actually you may do this a few times throughout the work, depending on task length).
10. Come back the next day and check over your work. A fresh look often reveals some things you missed before. Also fix any issues that you already realised in the interim (often you will think of things later).
11. If you created a branch, do whatever is appropriate for it. That might be merging it into the v11 branch, submitting a merge request on GitLab, or asking a senior developer to review then merge it.

Here are some general tips:

• It's sometimes a good idea to program algorithms in complete isolation. Separate the complexities of integration from the complexities of implementation. This also leads to better quality code because you will have been forced to abstract the problem already.
• If working with other remote developers, Cloud9 IDE is absolutely excellent. You can watch each other code and see the results of running your code.It is very useful for teaching programming technique, team debugging, and pair programming.

Debugging

When programming bear in mind that it is human to make lots of mistakes. Don't blame yourself for errors. The key is having a rigorous practices to identify and deal with them.

If you are stuck with a problem you haven't been able to quickly resolve, consider these logical steps for resolving it:

1. Output data involved in calculations so you can confirm you are calculating on the data you think you are.
2. Isolate the problem with a simpler test case that is safely and trivially repeatable. For example, if the problem is happening when submitting a form with a unique codename and an upload, make a test script that simulates the same environment but without you having to fill in the form.
3. Step through the code either using an IDE debugger (PHPStorm & xdebug), or simply by putting temporary var_dump and exit calls in the code.
4. At some point you'll narrow in on the problem, as you'll have inspected yourself at a point in the code where things aren't going as expected. Usually it is then relatively easy to solve the problem as you will have found a clue in the process, such as a non-true assumption you had previously made.

Here are some general tips:

• Always think of quick ways to prove/disprove theories, and check facts, like variables match the values you expect them to. If you narrow in on facts, you make solving a problem much easier.
• Get fully comfortable with the powerful debug tools in a browser such as Mozilla Firefox, such as: the DOM inspector, setting JavaScript breakpoints, setting to pause on JavaScript exceptions, typing into the console, watching the console for errors, and analysing network requests and responses.
• Get comfortable with unified diff files. They are very useful for many purposes.
• Have a mind-set of automatic tasks to increase your own efficiency. Write temporary scripts to get things done faster or to reveal data to prove/disprove hypotheses about the nature of a problem, or to build confidence that you've properly fixed things.
• Whenever something confuses you, try and understand your way through it, even if you've already found another solution. Confusion is an opportunity for learning, a revelation of a gap in your knowledge – make the best of it so that you get a strong understanding of all technologies. With time, things will get easier and easier, but only if you constantly work at expanding your understanding. Work to build up a unified mental model of how everything works and fits together across all of computer technology.
• If you want to see how a function is operating in random different calls (i.e. not always just the first call), add some temporary code like if (rand(0,10)==1) exit($some_variable); and refresh the page a few times. This is a useful technique for many cases where you need to sample data.
• PHP often will not tell you about syntax/parse errors in your code. If you are stuck with the code randomly exiting, try using php -l <filepath> to see if PHP's lint tool will tell you about a parse error. Use of the Composr Code Quality Checker tool is even more useful, returning all kinds of potential problems.

If you are debugging a standalone PHP script (i.e. outside Composr), make sure you have errors turned on:
Having full reporting of all PHP notices, and display of them on screen, helps greatly in finding causes of issues.

Debugging on live servers

Composr keeps a number of logs, which can be very helpful for debugging low-level back-end activities. Try and build up a model of what happened via careful analysis of logs, like Sherlock Holmes.

Useful Composr logs include:

• Run-time error logging (using the PHP error logging system) (data_custom/errorlog.php; can view from Admin Zone > Audit > Low-level logging)
• Logged page activity, available via 'Investigate user' on an IP address / member ID / username (useful also for finding all a member's recent IP addresses)
• eCommerce log (maintenance status), if enabled (enable it via creating an empty writable data_custom/ecommerce.log file; can view from Admin Zone > Audit > Low-level logging)
• System scheduler log, if enabled (enable it via creating an empty writable data_custom/cron.log file; can view from Admin Zone > Audit > Low-level logging)
• Health Check log, if enabled (enable it via creating an empty writable data_custom/health_check.log file; can view from Admin Zone > Audit > Low-level logging)
• Tasks log, if enabled (enable it via creating an empty writable data_custom/tasks.log file; can view from Admin Zone > Audit > Low-level logging)
• Permission checks, if enabled (useful for debugging why accessing is failing) (enable it via creating an empty writable data_custom/permission_checks.log file; can view from Admin Zone > Audit > Low-level logging); covered in the Access control and privileges tutorial
• Queries running (debugging), if enabled (enable it via creating an empty writable data_custom/queries.log file; can view from Admin Zone > Audit > Low-level logging)
• Disk profiling (debugging performance), if enabled (enable it via creating an empty writable data_custom/debug_fs.log file and using the keep_debug_fs parameter; can view from Admin Zone > Audit > Low-level logging)
• Profiling (debugging performance), if enabled (enable it via creating an empty writable data_custom/profiling*.log file; can view from Admin Zone > Audit > Low-level logging)
• Resource-fs operations (debugging and audit), if enabled (enable it via creating an empty writable data_custom/resource_fs.log file; can view from Admin Zone > Audit > Low-level logging)
• Processes and memory use tracking from calls to require_code (debugging), if enabled (enable it via creating an empty writable data_custom/require_code.log file, and specifying the GET parameter keep_show_loading_code=1 in a request; can view from Admin Zone > Audit > Low-level logging).
• Mail integration (debugging), if enabled (enable it via creating an empty writable data_custom/mail_integration.log file; can view from Admin Zone > Audit > Low-level logging)
• Banned IP address access log, if enabled (useful for analyzing hits to the software by banned IP addresses) (enable it via creating an empty writable data_custom/banned_access.log file; can view from Admin Zone > Audit > Low-level logging).
• HTTP requests log, if enabled, to view requests made outbound by Composr (enable it via creating an empty writable data_custom/http.log file; can view from Admin Zone > Audit > Low-level logging)
• REST endpoints log, if enabled, to view REST requests made to data/endpoint.php (enable it via creating an empty writable data_custom/endpoints.log file; can view from Admin Zone > Audit > Low-level logging)
• The mail log (view from Admin Zone > Audit > E-mail queue/log)

Other very useful logs:

• The web server's own access log; look carefully in it and you are likely to see things like HTTP response codes (e.g. 500 means Internal Server Error)
• Any errorlog files the web server may leave behind (native PHP error logs, which may or may not be enabled)

Correlate times and IP addresses to get a model of what happened.

Also the Compsor 'SU' feature (see the Testing access and privileges tutorial, Access control and privileges section) can be very useful for testing under different users without having to know their passwords.

If it is acceptable to debug code on a live server, putting exit and var_dump commands to only run for your own IP address can be useful (if (get_ip_address()=='...') { ... }). This is not recommended for a high-traffic site though, you're likely to make an occasional mistake that'll get spotted. So for this case, you'll just have to clone the database back to a development server and test like that.

Logging into live sites

It is common to need to log in as an administrator on a live site and not know the admin password (only knowing the hosting access).
If you add a line to _config.php like the following:
It will automatically log you in under the first administrator account. You can find your IP address from common websites.

Note that if you're using localhost, you'll probably be using ipv6, so you'll need to use the ipv6 local address, ::1.

Form fields

To save time and auto-populate form fields during testing, if you have the Web developers toolbar extension for Firefox, you can right click, then choose: Web Developer → Forms → Populate form fields.

Advanced deployment and customisation

Using Git to deploy

Deploying to a live server using Git is a great idea. However, be aware some caveats…

1. Disable dev-mode in the config file using:
   

   Code (PHP)

   $SITE_INFO['dev_mode'] = '0';
    
   git automatically enables dev mode, which intentionally interferes with things to help you develop quality code – such as adding extra checks, or disabling cookies so that you don't rely on them. This is great for development, where the developer is in control of everything, but not correct for production.
2. Use a decent .gitignore so you don't get a lot of noise when running git status. Our default one is good but may need tuning.
3. Either keep the config file out of Git, or put code in there to differentiate between machines via checking get_hostname().
4. Make heavy use of Git branches so that you don't get stuck having to perfect all code before you can update anything onto the server.

You can consider a half-way house, by having a full checkout on the server, then separately symlinking specific directories into the main public_html. This is useful if you only want certain directories in Git, such as a docs zone (docs in this example being managed offline, and pushed live using Git).

You can also consider multiple checkouts on the server, such as a testing site, and a live site. This allows you to test things with greater care.

Make sure your web server configuration blocks access to the .git directory. The default Composr .htaccess file should do this for Apache users.

Hidden features inside Composr

Composr contains a number of hidden 'values', 'keep' parameters, and empty file flags. These allow activation of special functionality that isn't considered important/mainstream enough to warrant user-interface space within Composr.

Empty files

The presence of the following empty files in the root directory have a special meaning to Composr:

• install_ok – don't complain if install.php is left present (DO NOT use this unless your install is not connected to the Internet or if you definitely have an install_locked file)
• install_locked – whether to lock the installer (prevent it running)

Other special files

If you put a file called closed.html then this will be shown to people rather than the normal closed-site screen, regardless of whether the site is currently closed or not.

Hidden 'values'

Values are like hidden configuration options. They are either hidden because they are managed by code (perhaps used for keeping track of something), or because they are obscure. To set a value, open up Commandr (Admin Zone > Tools > Commandr – or clicking the symbol at the bottom-left of any page) and type :set_value('<name>','1'); (replace '1' if appropriate, but usually we do use '1' to enable). In normal PHP code, you can use the same set_value function, and also the get_value function. You can automatically add a new hidden value just by setting it for the first time.

We have the following (maintenance status) which are either unlikely to be useful, or potentially unstable or unpolished:

• Integrations:
  • force_admin_auth – set this to '1' if you want any super-administrator to have to have some kind of authorisation-based login by redirecting them to http://yourbaseurl/admin_login/ which presumably you have configured to be a redirect-through that catches logins (Apache only)
  • disable_password_hashing – set this to '1' if you want passwords to be stored as plain-text in the database (not recommended)
  • webdav_root – set this to the subdirectory the WebDAV addon should run from. On Windows it needs to run from the root, i.e. '/'. However, to do this you would need to map onto a separate domain name. If you change this setting, you'll need to change the redirect in your .htaccess file (or non-Apache equivalent)
  • allow_member_mail_relay – set to '1' if the form_to_email.php script should allow e-mailing to arbitrary members via the allow_member_mail_relay parameter. This potentially allows people to spam your members, so you may want to add some kind of extra security here at a web-server level (i.e. restricting access to the script).
  • windows_auth_is_enabled – set to '1' to enable Integrated Windows Kerberos Authentication for automatic desktop client logins
  • postgres_fulltext_language – set to a PostgreSQL full-text configuration name for defining how to tokenise text (only set if you don't want 'english'). Existing indices would need hand-editing to the new value too.
  • disable_fulltext_sqlserver – set this to '1' if you are using Microsoft SQL Server and want to disable full-text search support. You may do this if you don't like the limitations of SQL Server full-text search (per-table searching, not per-field searching).
• Unofficial functionality that may or may not work correctly:
  • unofficial_ecommerce – set to '1' if your forum driver contains usergroup manipulation functions (we don't support this, but wanted to be able to allow those willing to extend forum drivers to use our full eCommerce framework)
• Workarounds for difficult server configurations:
  • disable_iconv – set this to '1' if your iconv extension causes PHP to crash, or if it is very slow
  • disable_mbstring – set this to '1' if your mbstring extension causes PHP to crash
  • prefer_curl – set this to '1' if the server requires you to not use network sockets directly from PHP and instead use the CuRL extension (although if you need this it is likely you actually need to just configure the proxy server options instead)
  • real_memory_available_mb – set this to the number of MB GD should consider available for image generation, if PHP is lying about its memory limit
  • http_faux_loopback – a regular expression of URLs that may be accessed via direct filesystem reading rather than HTTP loopback. Be very careful with this, as it could seriously harm security. It can improve performance however. Some poor webhosts may also require it (we've only seen one though). It does support PHP scripts, which are given special treatment, via passing through the php-cgi interpreter (file uploads and POST requests not supported). Set via a command such as: :set_value('http_faux_loopback','^'.preg_quote(get_base_url(),'#').'.*\.(allowed|extensions|separated|with|pipe)\??');. Note that there is a 255 character maximum for the expression.
  • disable_ssl_for__<domain> – set this to '1' to disable SSL certificate checking for the given domain name. This is not a great idea, but may be necessary if a required service has a broken certificate yet it only supports HTTPS. Note that this will not work on a Mac by default because Apple's build of cURL (using https://opensource.app…curl/lib/vtls/darwinssl.c) will use Apple SecureTransport instead of OpenSSL, which doesn't support disabling of verification
  • manualproc_mail – set this to '1' to debug problems with the PHP mail command (the PHP mail command won't relay error information from the sendmail process, this will replace it with something that does); it only works on servers that allow PHP process control
  • disable_cookie_checks – set this to '1' if you are somehow proxying traffic into your web server with an incorrect hostname header, so cookie checks must be skipped
  • disable_sitemap – set this to '1' if you need to disable Sitemap menus for any reason (perhaps it is too slow to generate or there is some kind of infinite loop bug that is not yet fixed). Set this to '2' to greatly reduce the depth of the Sitemap (this is probably what you want, as you don't want your admin menus disappearing).
  • avoid_register_shutdown_function – set this to '1' if register_shutdown_function is not reliable on this system (may appear to be unreliable on some Windows servers, for example)
  • http2_post_fix – set to 1 to delay an AJAX POST request while a warmup request runs; used to fix issue with HTTP/2 connections where write-back is broken
  • slow_php_dns – set this to '1' if the server has very slow DNS resolution on the web server's main network interface, and therefore to do DNS resolution via the command line instead
• Used internally by Composr:
  • cdn – this is used for storage of autodetected CDNs, prior to the config option for them being properly filled in
  • brand_base_url – the base URL to the brand site (by default this is https://composr.app but may be changed)
  • rebrand_name – if Composr is called something else as far as the site staff are concerned, set the name here
  • rebrand_base_url – to change where branding URLs go to (e.g. the stub that is used when linking to documentation), set it here
  • company_name – if Composr is being rebranded to be 'made' by another company, set the name here (this has no effect of copyright of course, but Christopher Graham allows this)
• Developer testing:
  • notification_safety_testing – set this to '1' to enable extra checks to ensure notifications or welcome e-mails don't get misdirected/over-sent
  • memory_tracking – set this to the number of megabytes peak memory usage after which you want the execution URL to be silently logged to the error log (useful for debugging memory usage on a live site). The value must be an integer, with no trailing 'M'. This does not impose a memory limit, which is 128MB by default in Composr, but may be turned off within certain carefully-chosen execution contexts (usually admin tasks which are necessarily memory consuming). Requires PHP 5.2+.
  • enable_profiler – set this to '1' if you want to enable the Composr profiler (maintenance status), which will write select performance information to data_custom/profiling.(ID).log. For your security you should ensure these files are web-accessible (we will do this by default if we can via the data_custom/.htaccess, but you should take care). The web server needs permissions to save files into data_custom/ (this is possible by default on modern suEXEC environments). Set to '2' if you want to read Linux performance data, which will then be included in the logging (takes a bit more work by the profiler, assumes Linux and sufficient permissions).
  • monitor_slow_urls – set this to a number of seconds to log any requests taking at least that many seconds to the error log
  • permission_log_success_too – set this to '1' if you want the permission log file (data_custom/permission_checks.log) to show both successful and unsuccessful permission checks. Usually it only shows fails.
  • disable_modsecurity_workaround – set this to '1' if you want to disable the ModSecurity workaround
• For system-integrators:
  • agency_email_address – set this to a secondary e-mail address where you would like all error e-mails to go (the primary one being the staff address). Only set this if you are an agency and want to receive error e-mails in addition/instead of your client.
  • page_template_always_ask – set this to '1' if Composr should always ask for a page template, even when clicking inline links to add pages
  • page_template_default – set this to the codename of a page template that should be the default
  • page_template_restrict_to_custom – set this to '1' if only custom page templates will be shown for selection
• For back-end developers:
  • textmate – set this to '1' if you are developing on a local machine and use the Apple Mac TextMate editor (maintenance status). It will cause TextMate editing links to come up in stack traces. You can also adapt other software to work with it, see the Emulating TextMate section.
  • allow_php_in_templates – set this to '1' to enable the undocumented Tempcode 'PHP' directive, and its short-hand syntax (written like normal PHP long-tags). This allows you to write PHP code inside templates, but please aware there are strong risks associated with this. It means that themes you install may contain PHP code, or people submitting Comcode content may put in PHP code which you could accidentally unwittingly activate when validating content (as it would be validated against your own security credentials). Note this option is not designed to work from Comcode, because Comcode will add HTML formatting around your PHP code, corrupting it (from a security point of view you need to be aware though, as that could be avoided with some care by a hacker)
  • git_autopull – set this to '1' if Git changes should automatically be pulled every minute (less frequently if the system scheduler interval is longer). Assumes a suEXEC-like server and a Git deployment and working shell_exec
• Interface tuning for usability:
  • disable_multi_homed_upload__cns_avatar – set this to '1' to not allow members to put in a URL for their avatar
  • disable_multi_homed_upload__cns_photo – set this to '1' to not allow members to put in a URL for their photo
  • disable_multi_homed_upload__<page-name> – set this to '1' to simplify the upload widget on that page (e.g. cms_downloads)
  • levels_to_expand__<ajax_tree-hook> – set some ajax_tree hook to auto-expand by a certain number of levels
  • no_confirm_url_spec_cats – set this to '1' if you don't want to have to confirm the category for a new catalogue entry when the category was specified in a URL parameter already
  • no_spec_cat__<catalogue-name> – set this to '1' if you don't want to have to confirm the category for a new catalogue entry when the category was specified in a URL parameter already
  • no_tech_login_messages – set to '1' if you don't want technical explanations attaching when a user is forced to login
  • disable_comcode_page_order – set this to '1' to disable the Comcode page ordering feature
  • disable_comcode_page_show_as_edited – set this to '1' to disable the Comcode Page "show as edited" feature
  • hide_import – set this to '1' to hide the import icon from the Admin Zone (as some other addons may depend on it, but you may not need an icon)
  • hide_zone_editor – set this to '1' to hide the zone editor from the Admin Zone (if you're not using side panels)
  • hide_polls – set this to '1' if you don't want to show polls in the CMS menu
  • hide_rss – set this to '1' to hide the RSS links from the Admin Zone
  • disable_bulkhelper – set this to '1' to disable the feature that asks you if you are bulk-adding records
  • disable_news_repimages – set this to '1' to disable rep-images for news categories
  • blogs_tab_enabled – set this to '0' for the Conversr member profile blogs tab to be disabled (if you want blogs, but just not on member profiles)
  • use_tos_lang – Set to '1' if you want your site to say "Terms of Service" opposed to "rules" when referring to the website rules (clear your language cache after changing this value)
• Low-level tuning:
  • commercial_spellchecker – set this to '1' if you want to enable the WYSIWYG editor's spellcheck.net spellchecker
  • disable_kid_gloves_html – set this to '1' if you want to completely disable the kid-gloves HTML escaping mode, i.e. avoid the need to make declarations in your overrides. You may want to use this if you override various files but aren't really writing custom code and don't really understand how to make the declarations
  • show_menu_items_as_url – set this to '1' if you want the menu editor to show URLs instead of page-links (they will then be converted back)
  • safelisted_blocks – set this to a comma-separated list of block names to safelist any user to use those blocks independently. This is useful for AJAX-loaded blocks on sites where users may leave pages open, letting their sessions or block authorisations expire. It also is a performance boost because it means that the run-time permissions don't need maintaining.
  • no_individual_gallery_view – set this to '1' if all images/videos in a carousel-mode gallery must be viewed from the carousel-mode view (never individually)
  • root_cat__images – you can set this to the codename of a gallery, which will be the root for gallery selection on the gallery addon's Add Image form (root is not shown as an option). Alternatively, it can be an Selectcode string.
  • root_cat__videos – you can set this to the codename of a gallery, which will be the root for gallery selection on the gallery addon's Add Video form (root is not shown as an option). Alternatively, it can be an Selectcode string.
  • comment_forum__galleries / comment_forum__images / comment_forum__videos / comment_forum__downloads / comment_forum__calendar / comment_forum__news / comment_forum__polls – override the comment forum for a particular content type. Make sure you refresh the page with the comment form in your browser after changing this, as the settings also get saved into the page HTML
  • comment_forum__catalogues__<catalogue-name> – define a comment forum for a specific catalogue
  • disable_required_lang_selection – set this to '1' if you want members to be able to leave their language selection as 'Unset' (otherwise normally it'll copy the auto-detected language into their profile or they can change it, but it never stays as unset). This option is useful on multi-site-networks, with each site having a different language
  • disable_base_check – set this to '1' if you do not want Composr to empty caches if the base URL gets changed (this is if the base URL is not defined, and someone accesses from something different)
  • pagination_when_not_needed – set this to '1' if you want the pagination wrapper template to load even if there is no pagination (useful for collecting result metadata via Tempcode)
  • catalogue_seo_source_map__<catalogue-name> – a comma-separated list of field IDs to use for SEO keyword generation (if not set, uses all fields). If all keywords in the field should be used, put an exclamation mark after the field ID
  • filtercode_protected_fields – a comma-separated list of fields Filtercode cannot work on (above the ones hard-coded as forbidden)
  • memory_limit – set this to a memory limit to override Composr's built-in one (e.g. "128M"). This will make memory bugs on your server more dangerous, so use at your own risk (a memory usage problem can take down a whole server) – although memory bugs are unlikely given that Composr users report if they ever see this error
  • canonical_keep_params – set this to a comma-separated list of keep_ parameters that should be allowed to propagate within search engines
  • default_event_type – set this to the ID of the default event type to select for new events
  • edit_with_my_comcode_perms – set this to '1' if content you edit should display with your Comcode permissions. Only enable this if you trust yourself to spot XSS attacks.
  • site_location – a real-world address for where your website is, sometimes used in metadata.
  • google_news_urls – set this to '1' if news URLs should contain a Google-News-friendly entry ID.
  • news_categories_blocked – set this to a comma-separated list of news category IDs if you do not want those to be selectable; useful for keeping legacy news categories out of the UI for the current editing team.
  • enable_delayed_inserts – set this to '1' to enable MySQL delayed inserts. Delayed inserts can cause a very small performance improvement if log calculations happen quite regularly; however they only work on MyISAM tables and are deprecated as of MySQL 5.6
  • disable_cannot_access_url_messages – set this to '1' to turn off attached messages about broken URLs that Composr tries to process
  • force_spell_checker – set to 'pspell' or 'enchant' to force that particular spell-checker, rather than auto-detecting
  • disable_handle_conflict_resolution – set this to '1' if you want to disable conflict resolution checking
  • disable_form_auto_saving – set this to '1' if you want to disable form auto saving
  • default_newsletter_mail_template – set to the name of a template to set the default template for newsletters (instead of MAIL)
  • default_comcode_text – set this if you want to have multi-line Comcode editing fields to have a default value; this is useful for setting default font styles but don't want to retroactively affect old content; if you want it to be able to auto-wrap quick-reply content then you can put {content} where you'd like the content to be inserted
  • addon_disabled_<addon> – set this to '1' to disable a particular addon (may not be supported by all addons)
  • mail_theme – set this to the name of a theme to force that theme to be used for all outbound e-mails. This is useful because otherwise the theme of the user triggering the e-mail to be sent will be used.
  • sitemap_orphans_to_pages – set this to '0' if you don't want orphaned pages to be put under the default Pages page grouping of the Sitemap
  • time_sensitive_rankings__<catalogue_name> – set this to a number for catalogue rating sorting to only consider this number of records. This is useful for a upvote/downvote system that is responsive to change in voting pattern. Requires the database to support window functions (MySQL 8+ does, Maria DB 10.2+ does).
  • keep_gallery_gps – set this to '1' if you want galleries (and related to feature) to not strip GPS metadata from images (which is done by default for privacy reasons)
  • hide_author_skills – set this to '1' if you do not want the author addon to have a skills field
• Low-level Action log tuning:
  • log_actions__create_topic – set this to '1' to enable Action logs for new topics
  • log_actions__reply – set this to '1' to enable Action logs for topic replies
  • log_actions__chat – set this to '1' to enable Action logs for chatroom posts
  • log_actions__im – set this to '1' to enable Action logs for instant messaging messages
• Low-level search tuning:
  • really_want_highlighting – set this to '1' if you want to force HTML to be converted to Comcode when searches happen, so keyword highlighting works better.
  • search_days__<search-hook> – set the default day search period for a search hook (e.g. 60, or blank for no limit). This may be set for reasons of user experience, or for performance – searching without a day constraint is actually faster.
  • fast_custom_index__max_ngram_size__<language> – Overrides the maximum ngram size config option for a particular language
  • fast_custom_index__enable_for__<search-hook> – Enable the fast custom index for a particular search hook (if you need more fidelity than the regular config allows)
  • fast_custom_index__enable_for__<language> – Enable the fast custom index for a particular language (if you need more fidelity than the regular config allows)
  • fast_custom_index__startup_hack – Forces the forum post indexer to resume where it left off. Useful when debugging indexing problems.
• Performance tuning (positive) [not official options as will break normal expectations of how the system will behave / complicate things]:
  • assume_indices_exist – set this to '1' to assume all indices exist, to avoid having to check before using them
  • lots_of_data_in_* – set this to '1', with '*' replaced with a database table name, if you want the Selectcode mechanism to work with recursive DB lookups rather than one huge flat lookup
  • disable_cat_cat_perms – set this to '1' if you want to disable catalogue category permissions
  • catalogue_limit_cat_field_load__<cataloguename> – set this to '1' if you want that catalogue to not have all fields loaded on category views, an optimisation that reduces templating flexibility
  • no_catalogue_field_assembly – set this to '1' if you want catalogue entries to be 100% hand-templated; removing this will improve performance but remove pre-assembly
  • no_catalogue_field_assembly_fieldmaps – see above, but only if fieldmap assembly not required
  • no_catalogue_field_assembly_grid – see above, but only if grid assembly not required
  • no_catalogue_field_assembly_tabular – see above, but only if tabular assembly not required
  • assume_modules_correct – set to '1' if module/block version checking should not happen; this will improve performance
  • no_priority_redirects – set to '1' if redirects should only be considered if a page cannot be found via other means
  • disable_tags – set to '1' if you don't want tags to be generated (small performance tweak)
  • disable_awards_in_titles – set this to '1' if you don't want awards to be displayed on content screens (small performance improvement)
  • disable_user_online_counting – set this to '1' if you don't want to count online users (small performance improvement)
  • disable_banner_count_updates – set this to '1' if you don't want to maintain banner view counts (small performance improvement)
  • pinpoint_submitban_check – set this to '1' if you only want submitter ban checks to occur in the CMS zone (small performance improvement)
  • disable_sitemap_for__* – disable sitemap generation for a particular page (e.g. if there's too much data in it and the server can't cope, and the data isn't so important anyway). Replace '*' with the page name
  • disable_view_counts – set this to '1' if you don't want view counts to be maintained (reduces database writes). You will probably want to disable the normal "Reveal usage figures" configuration option (Admin Zone > Setup > Configuration > Site options > Analytics / Logging) too, to hide all the zero counts. You can also set it to -1 if you do not want view counts to be recorded for staff
  • disable_normal_topic_read_history – set this to '1' if you don't want topic-read history to be maintained for things other than private topics
  • disable_member_tracking – set this to '1' if you want to disable the member tracking feature where it's supported (it shows who is viewing a forum topic, for example)
  • min_lastvisit_frequency – set this to the number of seconds waited between the last-visit time being updated (performance tweak, minimises database writes)
  • disable_flood_control – set this to '1' if you will not have flood control
  • implicit_usergroup_sync – set this to '1' to enable synching of implicit usergroups (see the Advanced Composr member system tutorial for more information)
  • shortened_tempcode – set this to '1' to not store parameter reference origins in Tempcode, which makes Tempcode use less memory/storage/CPU
  • aggressive_tempcode_compilation – set this to '1' if you want to compile some things aggressively into Tempcode that may occasionally change, such as member avatar URLs or resource URLs; setting this implies you will tolerate some outdated data in return for slightly faster cache rendering
  • save_jpegs_as_png – set this to '1' to allow Composr to detect when JPEG files would be smaller, and save JPEG data into PNG file extensions. It is a bit of a hack, and goes against standards, but works well
  • js_keep_params – handle keep variable propagation via JavaScript. Used in conjunction with $SITE_INFO['no_keep_params'] = '1';
  • slow_counts – set this to '1' if you have switched to InnoDB, as table row counts are slow and this switch enables an approximation instead in certain places. Do not use with other databases than MySQL or PostgreSQL or SQL Server.
  • innodb – set this to '1' if you have switched to MySQL's InnoDB
  • disable_post_rating – set this to '1' if you don't want to allow post ratings, which saves one query per displayed post
  • statistical_update_model – update view counts irregularly in bulk, using a statistical algorithm
  • safelisted_blocks – see explanation under "Low-level tuning"
• Performance tuning (negative):
  • flush_cache_on_action – set this to '1' to automatically flush the static cache when actions are performed
  • allow_admin_in_other_zones – set this to '1' if you want to allow admin and CMS modules to run in other zones (small performance cost)
  • memory_limit_simulate_hard – set this to '1' if you do not want the memory limit to be disableable; useful for testing compatibility with hosts that have low limits
  • disable_web_resources_static_compression – set this to '1' to not create .gz/.br versions of JavaScript and CSS files (such compression may slow a fast-changing development environment more than the gain in download speed helps)
  • cron_block_caching_careful_contexts – set this to '1' if you want Cron generated block-caching to use the requested theme and language to fill the cache; this requires loopback HTTP requests
  • force_memory_sort__<catalogue-name> – set this to '1' to force an in-memory sort on a particular catalogue (this is slower, but it will sort more accurately by resolving content language string fallback and Comcode fully before sorting)
• Easter eggs:
  • stupidity_mode – set to leet or bork, for a fun April Fools-style Comcode joke (clear the Comcode cache after you're done though)
• Experimental functionality
  • download_associated_media – set this to '1' to add an option on posting forms to have any referenced media automatically downloading locally, to protect against the risk of future broken links
  • highlight_extracted_file_data – set this to '1' if downloads searches should show highlighted data from extracted keywords, if there's no match on the body text of the download
  • download_gallery_queue_images – set this to a string of the number of images that exist in a gallery before gallery export triggers the task queue (if enabled)

Hidden 'keep' parameters

'keep' parameters are placed inside URLs in order to give Composr some extra information when loading a page. Their URL-presence is automatically relayed/preserved in all Composr links within the page. To enable a 'keep' parameter, simply append &<name>=1 to the URL (replace '1' if appropriate, but usually we do use '1' to enable). If there are no parameters in the URL and URL Schemes are not enabled, use a '?' instead of a '&'.

The 'keep' parameters available are:

• keep_cache – set to '1' to temporarily enable caching or '0' to temporarily disable. You can also use cache so it works on a per-page basis, with the exception of this not affecting the template and language caching.
• keep_cache_and_carry – set to '0' to disable the HTTP cache. Only works when logged in as a super-admin.
• keep_cache_avoid_for – set to a regular expression of codenames to not cache for (e.g. GLOBAL_HTML_WRAP|STANDALONE_HTML_WRAP would disable caching for the GLOBAL_HTML_WRAP and STANDALONE_HTML_WRAP templates; this is useful if smart decaching is either disabled or doesn't work because a template edited is included in another (smart decaching cannot handle this case; although you can also set $SITE_INFO['dependency__<directory>__<codename>'] = '[<codename>,]*'; settings in _config.php to inform the smart cache of dependencies)
• keep_dev_mode – set to '0' to disable development mode (which only runs if you're working out of a subversion repository anyway)
• keep_hide_dev_mode_message – set to '1' if you don't want the "Dev-mode is on" message to show if dev-mode is on (maybe you're taking screenshots?)
• keep_ext_check – set to '0' to force the webstandards checker to not check dependency files
• keep_force_htaccess – set to '1' to force the Conversr login to go via htauth in precedence over other login methods (e.g. login cookies, sessions)
• keep_simplified_donext – set to '1' to temporarily act as if the simplified do-next option is turned on
• keep_mobile – set to '1' to pretend you're using a PDA
• keep_has_js – set to '1' to force Composr to think you have JavaScript (useful if the has-JS cookie isn't saving yet you need to use a JS-based interface)
• keep_ldap_debug – set to '1' to forcefully show LDAP errors, when suppressing them might normally be wise due to over-sensitivity
• keep_show_query – set to '1' if you want the SQL query used by a search to be echoed out
• keep_just_show_query – set to '1' if you want Composr to echo out the query as above, but then to exit before running it
• keep_currency – set to an ISO currency code to tell Composr which currency you use
• keep_country – set to an ISO country code to tell Composr which country you're in
• keep_region – similar to above, but supports whatever regions are configured on the website
• keep_cat_display_type – set to a number representing the catalogue category display-type to try out (0=entry-tables, 1=lists, 2=matrix)
• keep_id_order – use this in conjunction with the admin_cns_groups module to force a usergroup reordering based on the named DB field of your choice
• keep_backup_alien – set this to '1' if the backup module should only be backing up non-Composr files (i.e. do a backup of the website, but not the software)
• keep_module_dangerous – set this to '1' to allow uninstallation of modules that are locked
• keep_preserve_ids – set this to '1' when doing an CMS-merge import, to preserve IDs in the import (and not import duplicated IDs); this is not supported officially
• keep_theme – set this to a theme name, to temporarily try out a different theme
• keep_safe_mode – set this to '1' so that where possible, any customised files will be ignored; the default theme will be used
• keep_fatalistic – set this to '1' if terminal warning errors should be handled as terminal fatal errors (which include stack traces), and '2' if you want to show extra detail (which may be risky, it could use a lot of server RAM); useful for debugging the origin of an error message
• keep_firephp – set this to '1' to dump permissions checks that occurred to FirePHP. It only works for authenticated administrators, but if the administrator is using keep_su to masquerade as another user it will work (which is particularly useful for debugging permissions problems).
• keep_firephp_queries – set this to '1' to dump queries that occurred to FirePHP. It only works for authenticated administrators, but if the administrator is using keep_su to masquerade as another user it will work. keep_firephp must also be set.
• keep_send_immediately – set this to '1' if you want to debug newsletter code, and not have newsletters sent in the background
• keep_su_online – set this to '1' if you want the user keep_su is used with to also show as online, and for their last online/submit statuses to be updated
• keep_su_strict – set this to '1' if to strictly masquerade as another user rather than to allow a few exceptions like the display of stack traces
• keep_minify – set this to '0' if you want to use a JavaScript debugger and hence do not want the JS to be minified (unreadable), or otherwise are debugging and don't want HTML/CSS/JavaScript to be minified or a CDN to automatically be used
• keep_show_loading – set this to '1' if you want comments inside the HTML source showing memory usage taken up as major server-side dependency files load
• keep_show_loading_code – set this to '1' if you want comments inside the HTML source showing memory usage taken up as server-side PHP scripts load
• keep_memory_limit – set this to '0' to not put on a memory limit (admins only); using this will also add in the option to remove paginations, for those with the remove_page_split privilege
• keep_memory_over_speed – use less memory for Tempcode
• keep_query_limit – set this to '0' to not place a query limit during development mode
• keep_on_same_msn – set this to '0' if you are doing a Composr site merge import and want to force Conversr content to import even if the importer thinks that you are on the same MSN as the other site
• keep_theme_test – set this to '1' if you are testing lots of themes. It will cause installed theme addons to have their Comcode pages extracted with a special prefix, it will remove that prefix on addon packaging, and page detection will give preference to the prefix for the particular theme you are currently using (so use in conjunction with keep_theme).
• keep_debug_notifications – set this to '1' if you are debugging the notifications system and therefore need them to be sent before the page output finishes
• keep_debug_tasks – set this to '1' if you are debugging the tasks system and therefore need them to be run immediately (i.e. without going into the task queue)
• keep_devtest – used during development mode to test URL building is working correctly, of no use manually
• keep_debug_has_cookies – set this to '1' to tell Composr cookies are supported when JavaScript may be off
• keep_memory_limit_test – set to a number of megabytes under 32 to reduce the memory limit to this value; useful for performance debugging
• keep_realtime_test – set this to '1' for testing the realtime feature's display (it injects random data)
• keep_wide – set this to '1' to remove panels
• keep_wide_high – set this to '1' to remove panels, as well as the header and footer
• keep_rating_test – set this to '1' if you are testing rating and don't want to be locked out for having already rated
• keep_all_cpfs – set this to '1' if you want to edit bundled CPFs that aren't currently marked as editable
• keep_oembed_cache – set this to '0' to disable the oembed URL cache temporarily, when debugging oembed endpoints
• keep_url_monikers – set to '0' to disable URL monikers
• keep_url_scheme – set this to '0' to temporarily disable URL Schemes. This is useful for working out page-links/match-keys
• keep_force_open – set this to '1' if you want to ignore a closed.html file and get to the website
• keep_frames – set this to '0' to disable optional iframes / AJAX frame-simulation
• keep_failover – set this to '1' or '0' to force failover mode to on or off
• keep_grow_template_meta_tree – set this to '1' for the "Grow template meta-tree" configuration option (Admin Zone > Setup > Configuration > Performance options > Data gathering) to be considered as temporarily enabled
• keep_has_cron – set this to '1' or '0' to override UI adaptations to the system scheduler being on or not (e.g. to enable scheduling options as if the system scheduler was on, even if they won't run)
• keep_smart_decaching – set this to '1' for smart decaching to be enabled, even if disabled in _config.php (it is on by default, so you may want to use this flag if you disabled it but want to ease your development workflow)
• keep_infinite_scroll – set this to '0' or '1' to temporarily change the infinite scrolling setting
• keep_ecommerce_local_test – set this to '1' to test local payment in dev-mode (without needing SSL)
• keep_type_strictness – set this to '0' to disable developer type strictness checks on ocProducts PHP
• keep_xss_detect – set this to '0' to disable developer XSS detection on ocProducts PHP (this has nothing to do with live security, it just aids debugging)
• keep_ecommerce_local_test – set this to '1' to test local payment in dev-mode (without needing SSL)
• keep_test_version – set to a dotted version number to test the version block in the Admin Zone dashboard as if you're running a different version. Likely needs combining with keep_cache_blocks=0
• keep_notifications – set this to '0' to suppress sending of notifications (only works if logged in as an administrator)
• keep_quick_hybrid – set this to '1' in a quick installer to let the installer read code files from disk while installing if they are there; this allows hot-fixing/hot-debugging of installation bugs
• keep_show_timings – set this to '1' for the installer to spit out some timings information (for performance debugging) and turn off auto-continue
• keep_translate_sort – set this to 'recent' to make the content translation interface show recent strings first; useful during debugging
• keep_debug_fs – set this to '1', while in the installer, or logged in as an admin, for disk I/O to be slowed and file system case sensitivity to be checked. Or, to a higher number of milliseconds if you want to introduce fake I/O latency. It is for developer filesystem testing.
• keep_manual_import_config – set this to '1' in the import module to disable automatic detection of the import target's database settings from its filesystem config file (if you don't have it, for example)
• keep_fast_custom_index – set this to '0' or '1' to override which search engine to use, where possible; 0 for the database's own full-text search, 1 for the fast custom index
• keep_check_backdoor_ip_dns – set this to '1' to check against hostnames in the backdoor_ip setting
• keep_timer_display – set this to '1' if you want quiz timers to show in the quiz module, even to users with the bypass_quiz_timer privilege
• keep_expand_sitemap – set this to '1' for all nodes of a sitemap menu to be expanded, with no expand/contract icons (useful for mass-opening all pages on the Sitemap)
• keep_import_tick_all – set this to a base directory to import and it will auto-populate that directory and make the importer pre-tick (check) all the import types, for fast import testing of smaller databases

The following 'keep' parameters have interface triggers, but are also handy for manual activation:

• keep_markers – set this to '1' if you want template start/end markers to be output in the page HTML source when viewed in Composr
• keep_template_magic_markers – set this to '1' if you want specially encoded "invisible" template start/end markers, that the template editor can pick up on
• keep_webstandards_check – set this to '1' to perform advanced output checking (for staff only), giving errors if technology guidelines are broken (such as XHTML5, CSS, and WCAG). If you enable this flag you will be informed of any accessibility problems on your website, but it severely effects performance as well as your administration experience.
• keep_su – set this to a username or member ID when logged in as admin, to pretend to be that member. May choose the guest member ('Guest' on Conversr by default).
• keep_theme_seed – set this to kiddie or random or a 6-character HTML colour code, to get dynamic themegen going for your page view (this is very slow, but fun!); you must be staff, with a confirmed session, for this to work
• keep_theme_dark – used in conjunction with keep_theme_seed; set to '1' if it is a dark seed
• keep_theme_source – used in conjunction with keep_theme_seed; set to the name of the theme being used as the source
• keep_theme_algorithm – used in conjunction with keep_theme_seed; set this to 'equations' or 'hsv' to set the theme algorithm
• keep_print – set this to '1' to display a 'printer friendly' version of the page
• keep_lang – set to a two-letter ISO language code to tell Composr which language to use
• keep_session – this isn't useful for manual editing, but it is used to note session-IDs when [session-]cookies are not working
• keep_timezone – this is the timezone code (tz style) to view in
• keep_catalogue_<catalogue-name>_root – set to a category ID for the current viewed content type to trick the breadcrumbs into showing a 'virtual root'; this can also be activated by browsing to the category you'd like to be virtual root (as staff) and then clicking on the final link in the breadcrumb chain (the link representing the current category)
• keep_wiki_root – as above, but used to propagate Wiki+ virtual roots
• keep_gallery_root – as above, but used to propagate gallery virtual roots
• keep_download_root – as above, but used to propagate download virtual roots
• keep_forum_root – as above, but used to propagate forum virtual roots
• keep_page_root – as above, but used to propagate Comcode page virtual roots

The following 'keep' parameters have interface triggers and are not useful for manual use:

• keep_full_structure – whether to avoid merging nodes together in the Sitemap, used by the Permissions tree editor so that permissions on non-accessible nodes are settable
• keep_step8_all_at_once – whether to process step 8 of the installer in one execution rather than breaking it up into multiple executions (to avoid server time-outs), used by headless install code as it does not support this behaviour

The following aren't 'keep' parameters, but are still useful and otherwise behave in the same way:

• auth – set to '1' to make Composr request HTTP authentication. This is useful for RSS URLs, if you want the feeds to be generated as authenticated.
• wide – set to '1' if you don't want to show side panels
• wide_high – set to '1' if you don't show to show panels or the TOP/BOTTOM
• start – set to the start number in the result browsing
• max – set to the maximum number of results to show per-page
• page/type/id – standard URL routing parameters
• kfsXX – this isn't useful for manual editing but people ask what it's for; it's to preserve the browse positions in forumviews, such that when returning, the browse positions are remembered
• Tracking:
  • _t – used for tracking referral codes, useful for marketing campaigns, split testing, and referral schemes
  • _lead_source_description – used for tracking where a lead came from within Composr, when it's going to be sent off to an external CRM (via some non-bundled addon)

Extra _config.php settings

Setting,
in _config.php will disable e-mails. This is useful on a development server running with live user accounts.

If you use the config_editor.php script to edit the base configuration then you'll find lots of other settings to play with.

Providing default parameters to forms

Many forms in Composr accept GET parameters for specifying default field values. This includes most 'add' forms in the system. Generally the GET parameter is exactly the same as the actual field name in the HTML, so it's easy to work it out.

We also have some special cases coded up directly. These are often referenced between different parts of the Composr code. The details below are divided by module, then type, then parameter name.

• topics
  • new_post
    • quote
    • whisper_to_member
• warnings
  • add
    • member_id
    • post_id
• admin_aggregate_types
  • add
    • aggregate_type
  • sync
  • type
• admin_ecommerce_reports
• add
  • group_id
  • trigger
  • type_code
• admin_cns_forums
  • add
    • parent_forum
    • forum_grouping_id
• admin_cns_merge_members
  • browse
    • from
    • to
• cms_authors
  • _add
    • author
• cms_banners
  • add
    • b_type
• cms_blogs
  • add
    • cat
    • validated
• cms_booking
  • add_booking
    • bookable_id
    • day
    • month
    • year
• cms_catalogues
  • add_entry
    • catalogue_name
    • category_id / category_id_suggest
    • field_*
    • validated
  • add_category
    • catalogue_name
    • parent_id
  • add_catalogue
    • id
• cms_calendar
  • add
    • date
    • e_type
    • validated
• cms_downloads
  • add
    • cat
    • validated
  • add_category
    • parent_id
• cms_galleries
  • import
    • member_id
    • cat
  • add
    • cat
    • validated
  • add_other
    • cat
    • validated
  • add_category
    • parent_id
• cms_news
  • add
    • cat
• cms_quiz
  • add
    • validated

All default parameters are flagged as 'non-canonical' to search engines, so that the search engine knows that variations in value are not indicative of entirely separate web pages.

Overriding files

If you change an original Composr file for a client project you must save it into the _custom equivalent of its original directory. E.g. if you change site/pages/modules/calendar.php you need to save into site/pages/modules_custom/calendar.php. Otherwise if the client ever upgrades their website chances are all the customisations you've made would be lost when a newer version of the original file is placed.

This said, It's best to avoid completely overriding default files if possible, so that any bug fixes from upgrades are more likely to work. Composr provides some techniques so you can override just stuff you want to change rather than a whole file.
This is explained in the The Composr programming framework tutorial (there are two techniques: replacing/supplementing, and programmatic alternation).

If you are considering overriding a file just so that you can add some new functions to it used by one of your new modules (for example, adding a new form_input_something function), consider adding to a new file (or existing file in your module's addon) instead. There is no rule that says similar functions have to go together, and it makes no sense unless they are all core functions – in fact it increases average memory usage to have excess functions defined in core files (each required-up function uses quite a lot of memory).

Hints for developing websites for other people

Composr is a great platform for web developers. A web developer can build all kinds of sites using Composr – from simple new media sites, to social networks, to seemingly-bespoke sites such as property directories.

However, if you're making a website for somebody else it is important to bear in mind they probably do not want to learn Composr like you have.

Here are some assorted hints:

• You might want to use the debranding feature, Admin Zone > Style > Debranding
• You might want to give the user a non-admin username (e.g. a super-moderator user instead), and then set Admin Zone permissions so that many unnecessary pages are hidden. Composr is then smart enough to automatically simplify its interface to compensate.
• When deploying make sure to get your canonical URL right. Ask the client whether they want 'www.' in their URL or not. Set up a .htaccess file (https://css-tricks.com…pets/htaccess/www-no-www/) (or non-Apache equivalent) that redirects the non-canonical URL to the correct one. This will remove a lot of headaches related to duplicate cookies, bad SEO, confusion, and Composr error messages.
• Always, always, test the e-mails a site gives out. It is very embarrassing if you accidentally customise your MAIL.tpl template in a custom theme and then find e-mails are being sent out of the Admin Zone using the default theme and Composr logo. Always save your MAIL.tpl in the default theme, and make sure that the default theme's 'logo/-logo' and 'logo/standalone-logo' theme images correctly reflect your design. And, test it! If you made complex changes, test in different e-mail programs/webmail-apps.
• Develop using a staging/developer website, and deploy to live/customer-demo after you've finished all changes. This primarily involves uploading changed files. Composr has a great tool for transferring data between Composr sites using XML too, if you find you are needing to sync data.

Considerations for design and themeing:

• If there are separate frontend and backend developers, the frontend developer can do an effective mockup by coding a simple mini-module/mini-block that uses a static array of data passed directly into templates. This way the built CSS and markup is "Composr ready" (particularly things like the 'IMG' symbol can be used from the start). This requires the frontend developer to have basic PHP and Tempcode experience.
• In particular, do not make theme image changes using Composr's themes editor, as these are really painful to sync. Make use of Composr's automatic theme image detection by simply saving new theme image files into appropriate directory locations; note you will need to clear the caches if you are overriding a default image.
• If you are adding new functionality and write new templates, it is best to save these into the default theme's templates_custom/javascript_custom/xml_custom/text_custom/css_custom directories, so that your functionality works on all themes. If you are overriding default theme templates/CSS, it is then that you save them into your site's theme. A similar rule applies for custom images.

Take particular care with Comcode pages:

• If for some reason you don't want the WYSIWYG editor to run for a page (e.g. if it has a Google ad on, that can cause the editor to malfunction) then include the following hint in it: {$,page hint: no_wysiwyg}
• Remember when uploading to a live site that you do not want to overwrite customised Comcode pages, or other files that may have since been changed. Be careful what you upload, and keep backups.

It's overkill to apply the full range of Composr coding standards if you're writing PHP code for use on just one website. You shouldn't get into bad habits by skipping any of the conventions but some of our standards are particularly time consuming to meet, and in particular the following time savers can be applied:

• It is better to have "hackerish" overrides (using the code rewrite facility) than to override whole files. This is because it makes upgrades easy, which is far more important than code maintenance for a one-off Composr project.
• You can almost always assume the site runs one theme and thus save your new templates into the new theme. One caveat is the 'CMS' zone will need to run your theme, and also bear in mind that any files used in outgoing e-mails must be available from the default theme.
• You don't need to write everything using language strings.
• You don't need full MVC. For example, you could code in add/edit functionality straight into modules rather than via a sources file (model).
• You can make more assumptions than you ordinarily would be able to. For example, you might be able to assume cURL is installed. It is sensible to document your assumptions though, to make sure they continue to be met, or so someone breaking them has an idea that they need to act to change your code.
• For simple dynamic functionality you can use mini-modules and mini-blocks instead of full blown object-orientated modules and blocks.
• You do not need to write PHPdoc comments unless you think other developer's would benefit from them.
• On some sites you may be able to assume that accessibility isn't an issue (e.g. on a video site it's unlikely to need to cater for blind users – unless you plan to support audio description). Generally we would advise to be very cautious here, it usually does not take much longer to support full accessibility, and is worth doing routinely.

Bear in mind that some sites need maintenance or have multiple people working on them, so you should bear this in mind when you decide the level of engineered design and code documentation to implement for your custom functionality. It is a trade-off between initial development cost, and long-term time savings.

Scripts and tools

There are a large number of scripts and tools for helping in particular situations, outside the normal content management processes of running a Composr website.

Many of these are designed just for programmers, to speed up development. But some are particularly useful just for basic configuration management and DevOps.

In this section we document all the scripts and tools that may be generally useful to you as a developer, with a few categories of exceptions:

1. Commandr commands (go to Commandr and type commands to see a list)
2. Cleanup tools (go to Admin Zone > Tools > Cleanup tools)
3. Tools within the upgrader (upgrader.php)
4. Scripts with code listings provided in the Professional upgrading tutorial
5. IP address lookup (when you do an Investigate User in Composr you get various links to do investigations on the IP address like geographic lookup)

This section is not designed as primary documentation for the tools. Those tools not designed for programmer use may be described in various individual tutorials. The tools for programmers are typically undocumented, but a programmer can analyse the code to see how they work (and make improvements as needed).

Website management

Filename	Purpose	Addon	Call method
decache.php	Clear out file-system caches, and leave a mark for the next Composr page load to clear database caches.	helper_scripts	Command-line via PHP interpreter
fixperms.php	Set appropriate file permissions.	helper_scripts	Command-line via PHP interpreter
data/failover_script.php	Continually check system load, and initiate/deinitiate failover mode as required.	failover	Command-line via PHP interpreter + 'screen'
adminzone:static_export	Export website as a static site, as far as is possible.	static_export	Composr module
data/comcode_convert.php	Do various conversions to and from Comcode.	core_rich_media	Web browser
data/cron_bridge.php	Run the system scheduler hooks. Can also be used to probe for the likely Cron command to configure against.	core	Cron (or equivalent) via PHP interpreter
upgrader.php	Performing upgrades, detecting alien files, missing files, corrupt files, missing permissions, clearing the caches (without having to run the full Composr), deleting files accidentally uploaded from non-installed addons.	core_upgrader	Web browser
data/commandr.php	Command line tunnel into Commandr.	commandr	Command-line via PHP interpreter

DevOps

Filename	Purpose	Addon	Call method
delete_alien_files.php	Provide pasteable commands for deleting non-bundled addons, and alien files not related to installed addons. To be honest you probably are better off using the integrity checker from the upgrader, copy & pasting and changing into terminal commands via a regexp. Never-the-less this script may be useful for some people, especially ones wanting to take it as a starting point for something more custom.	meta_toolkit	Command-line via PHP interpreter
db_export.sh	Export the live database to an SQL dump.	helper_scripts	Command-line via Bash
db_import.sh	Import an SQL dump to the live database.	helper_scripts	Command-line via Bash
db_init.sh	Create an empty database as specified by _config.php.	helper_scripts	Command-line via Bash
adminzone:tar_dump	Save the live website into a TAR file.	addon_publish	Composr module
adminzone:sql_dump	Save the live database into an SQL file.	meta_toolkit	Composr module
recentchanges.sh	ZIP up recently changed files.	helper_scripts	Command-line via Bash
data_custom/cleanout.php	Customisable script for erasing all Composr data. Useful for undoing a test import on a new site, or removing test data before going live. Only to be used by programmers who understand how to customise the code to specify what to get deleted.	meta_toolkit	Web browser
sources/critical_errors.php	Monitors for logged critical errors, and e-mails them to site staff.	core	Command-line via PHP interpreter + 'screen'
data_custom/compile_in_includes.php	Compile Composr overrides against originals, for a slight performance improvement	performance_compile	Command-line via PHP interpreter
data_custom/health_check.php	Run a Health Check.	health_check	Command-line via Bash or Web browser

Theme development

Filename	Purpose	Addon	Call method
:theme_debug	Tool providing a number of fixes to themes to get them up to spec.	theme_debug	Composr module
adminzone:css_check	Search for possible CSS problems.	theme_debug	Composr module
themechanges.sh	Show a diff of theme changes.	helper_scripts	Command-line via Bash

Development

Filename	Purpose	Addon	Call method
_tests/codechecker/codechecker.sh	Run the Code Quality Checker.	testing_platform	Command-line via Bash
line_count.php	Count the number of lines of code, excluding third-party code.	meta_toolkit	Command-line via Bash
adminzone:admin_generate_adhoc_upgrade	Generate a TAR file of recent central repository file changes to upgrade a site's installed addons. Works by looking at what exact addons are installed on the site, and finds if they are outdated.	meta_toolkit	Composr module

Core development

Filename	Purpose	Addon	Call method
_tests/codechecker/build.sh	Build the Code Quality Checker Java code.	testing_platform	Command-line via Bash
_tests/codechecker/phpdoc_parser.php	Build the function signature library used by the Code Quality Checker.	testing_platform	Command-line via PHP interpreter
data_custom/build_rewrite_rules.php	Rebuild all files containing rewrite rules.	cms_release_build	Command-line via PHP interpreter
data_custom/transifex_pull.php	Pull language packs down from transifex.	addon_publish	Command-line via PHP interpreter
data_custom/transifex_push.php	Push language packs up to transifex.	addon_publish	Command-line via PHP interpreter
template_xss	Look to see if templates may have XSS holes. Can be given a parameter to write "fixes" back in (which should be verified by hand).	testing_platform	Automated test
adminzone:string_scan	Tool to work out whether language strings are administrative or not.	meta_toolkit	Composr module
adminzone:plug_guid	Insert GUIDs for all do_template calls that don't currently have them.	cms_release_build	Composr module
data_custom/stress_test_loader.php	Create a large amount of test content.	stress_test	Command-line via PHP interpreter
data_custom/change_addon_bundling.php	Move an addon between bundled and non-bundled.	addon_publish	Web browser
data_custom/composr_mobile_sdk_build.php	Bundle string and image assets from a Composr website for making a mobile app.	composr_mobile_sdk	Command-line via PHP interpreter

Documentation development

Filename	Purpose	Addon	Call method
adminzone:sql_schema_generate	Build structure (schema) documentation for the Composr database.	meta_toolkit	Composr module
adminzone:sql_schema_generate_by_addon	Build structure (schema) documentation for the Composr database, by addon.	meta_toolkit	Composr module
adminzone:sql_show_tables_by_addon	List all the tables in all addons.	meta_toolkit	Composr module
adminzone:doc_index_build	Build the documentation index page.	composr_tutorials	Composr module

Release management

Filename	Purpose	Addon	Call method
adminzone:build_addons	Mass-build addon TARs.	addon_publish	Composr module
adminzone:publish_addons_as_downloads	Publish addon TARs on the website the module runs on.	addon_publish	Composr module
adminzone:admin_modularisation	Ensure consistency between files, package comments, addon info, and addons' file lists. The UI offers automatic fixes for some issues. Note that this tool is intentionally aggressive, and some issues may not be appropriate to fix.	cms_release_build	Composr module
adminzone:admin_make_release	Make a Composr release.	cms_release_build	Composr module
adminzone:_make_release	Publish a Composr release on the website the module runs on (not usually run directly, linked to from adminzone:make_release).	cms_homesite	Composr module
adminzone:push_bugfix	Push a bug fix to composr.app.	cms_release_build	Composr module

API services

Filename	Purpose	Addon	Call method
data/endpoint.php	JSON web service entry point.	core	Web service call
data/webdav.php	WebDAV entry point.	webdav	Web service call
backend.php	RSS/Atom entry point.	syndication	Web service call
data/form_to_email.php	Pipes POSTed data to a staff e-mail.	core_feedback_features	Web browser POST target
data/opensearch.php	OpenSearch provider, for search auto-completion and initiation.	search	Web service call
data_custom/user_export.php	Export users to a CSV spreadsheet file, designed for automatic synching.	user_simple_spreadsheet_sync	Web service call
data_custom/user_import.php	Import users from a CSV spreadsheet file, designed for automatic synching.	user_simple_spreadsheet_sync	Web service call

Composr version naming

In order to have a consistent upgrade system, Composr versions are named according to a rigid scheme.

For written version names we use the "pretty" naming scheme. This is best shown by example…

The following are examples of valid "pretty" names:

• 3
• 3.1
• 3.1.4
• 4 alpha1
• 4 beta1
• 4 RC1
• 4.3 dev1734368937

The following are examples of invalid "pretty" names:

• 3.0 [reason: leading ".0" components]
• 3.0.0 or 3.1.0 [reason: leading ".0" components]
• 4alpha1 [reason: missing space]
• 4 alpha 1 [reason: extra space before "1"]
• 4Alpha1 [reason: alpha/beta must always be lower case]
• 4 RC 1 [reason: extra space before "1"]
• (anything with dashes in)
• (anything with 'final' or 'gold' in)

Rules:

• Versions should never be skipped (except for dev build numbers). E.g. There can't be a 3.2 if there is no 3.1.
• Versions should never be repeated. You must always bump the major, minor, patch, or qualifier number on every release.
• Do not pull and re-release a version if it has already been released publicly (e.g. if you publicly release 3.0 and find a critical issue, do not pull 3.0; instead, release 3.0.1 with the fix and indicate on the 3.0 download / news post not to use that version).
  • Exception: Pulling alpha versions and re-releasing them is allowed (but not recommended) because we do not support alpha versions in the release/upgrade cycle.
• All versions have to have formally named news posts, in the 'New releases' news category. Our upgrade scripts do this automatically.
• Dots are not decimal points. They split major/minor/patch levels. You can have numbers in any dot section exceeding 9 and going to 10.
• The term "alpha" means not all planned features have been implemented. Bugs are highly likely. Should solely be used for testing. And those using alpha versions should expect to make clean installs each release.
• The term "dev" means an alpha build that is not to be publicly released or put into the release cycle. These builds are for testing only (as a fresh install) and cannot be used in the upgrader. Unlike alpha/beta/RC builds, dev builds should have an epoch timestamp as the build number (e.g. dev1734368937).
• The term "beta" means bugs are anticipated / the release did not meet Composr's standards of testing. Not recommended for live / production sites.
• The term "RC" (release candidate) means significant bugs are not anticipated, but we are not yet satisfied with the level of testing done on the release. Live / production sites may use these versions with caution.
• dev/alpha/beta/RC is technically attached to a patch-level release, not a major/minor version. i.e. any patch release can have dev/alpha/beta/RC.
• The "minor" version should be bumped whenever implementing new / significant features, but are unlikely to break sites from previous minor versions.
• The "major" version should be bumped whenever significantly changing the core Composr APIs to the point sites on previous versions need to use extra care / testing / planning when upgrading.
• Otherwise, the "patch" version should be bumped, or if an alpha/beta/RC tag has a number on it, bump that.

Inside our code we always use the "dotted" scheme, which is the same except that the space is replaced by a dot.

See the notes at the top of sources/version2.php for a full run-down of our ways of expressing versions.

Conclusion

We're glad you've made it to the end of our Code Book. Thanks for reading this far, and we hope you find Composr an effective and enjoyable environment to code for.

Feedback

Have a suggestion? Report an issue on the tracker.