Composr Tutorial: Statistics for your site (analytics)

Lots of statistics types are available

(Click to enlarge)

It's important to be able to see graphs and summaries of your site's activity: average visitor counts, which search engine terms are finding your site, the browsers your visitors are using, and so on. Composr has a sophisticated statistics system, documented in this tutorial.

Google Analytics vs Composr statistics

Google Analytics is extremely popular free tool for website owners and marketers for viewing website statistics. For that reason, I am starting the tutorial by directly addressing this.

We intentionally do not try and 'compete' with Google Analytics. There is significant overlap but Composr statistics and Google Analytics are complementary tools. Here are some key differences:

• Pros for Google Analytics:
  • Google Analytics is a very sophisticated and complicated product that provides a lot of functionality to professional marketers. Composr statistics is fairly straight-forward.
  • Integration with other Google products, such as Google AdSense (to see what is producing AdSense income, for example).
  • Integration with Google Search Console (for SEO hints).
  • Google Analytics has access to insights from Google's own marketing. For example, it can show statistics of genders of visitors.
• Cons:
  • Google Analytics ultimately only knows about web visits to URLs, unless you do custom programming or complex configuration. Composr statistics has much more contextual information (e.g. knowing how Composr pages work), and can draw graphs for things beyond just web visits (e.g. sales).
  • Google Analytics uses JavaScript to initiate tracking. Composr does tracking server-side, not relying on JavaScript, and thus not subject to web browser configuration/blocking. While Composr by default does require JavaScript to display pages, views will still be logged even if JavaScript is not enabled by users.
  • Google Analytics uses tracking cookies, which is a privacy concern and questionable under legislation such as GDPR. Composr doesn't use anything beyond regular IP addresses, session IDs, and member IDs.
  • Google Analytics is a product of a large multi-national corporation that makes money through targeted advertising, and has a number of major controversies around it (e.g. it is highly probable your analytics data is being sold to third-party data brokers or used for targeted advertising in Google AdSense). Composr statistics is hosted only on your own website and under your complete personal control.
  • Google Analytics is closed source. Composr statistics is just another bundled Composr addon, and therefore fully Open Source.

Basic users may be happy with only Composr statistics, especially given the privacy and freedom around big-tech marketing. Expert marketers will likely want the power of Google Analytics, there is no doubting it is an extremely powerful tool and a professional standard.

Google Analytics is easy to set up in Composr as we have support built-in: you just need to enter your analytics code in the Composr configuration.

The administration module

Installing geo-location data (maintenance status)

Once the geo-location data is installed, all the extra statistics can be used.

Categories and Graphs

The statistics module splits up the graphs into a number of categories. The exact categories and graphs available depend what addons you have installed, and to some extent your configuration – at the time of writing, there are over 65.

There are a large number of different graphs available for viewing, and most of them function in an obvious way. This documentation section will highlight some specifics that are not so obvious.

Tally by

You can set time-based graphs to tally records (pivot) in different ways. There are 2 kinds of tallying:

1. time series – this groups all records into chunks of continuous time, and puts the tallies on a graph
2. flow pattern – this groups all records by repeating patterns of time; for example, you can show a tally of all the records that happened at 2pm (regardless of date)

Time series show as a line graph, while flow patterns show as a bar chart.

Audience: demographics

Demographics graphs give you an idea who your users are. It may help you have an idea who you need to work harder for – be that attracting new audiences, or doubling down on what you now understand as your core audience.

At the time of writing, only age and requested language are available analytics. This is because these are the only built-in pieces of demographic information Composr gathers from members. Gender, for example, is not a built-in field but can rather be implemented as a Custom Profile Field (which statistics do not track). The same is true for location fields. You could write your own statistics hooks to track CPFs if you desired.

Audience: technical

These graphs get into the more technical aspects of your audience, such as what devices they are using to access your site, what operating systems they run, and more. You can utilise these statistics to make decisions, for example, on your theme's design and behaviour to accommodate the most people.

Content growth

This category presents two different ways to measure your site's content growth over time:

• Action logs, which is primarily an indication of how often content is getting added, updated / revised, and removed (unlike the Action Logs graph in Feedback and engagement, this graph focuses specifically on content-based actions)
• sitemap, which is primarily an indication of physical content growth on your site

Conversions

This is a very important category which indicates when visitors show interest (or lack thereof) in your site / company / products. There are various activities that indicate a visitor's interest:

• when they sign up for your newsletter
• when they create an account (joining)
• when they purchase a usergroup subscription
• when they click on tracking links / codes you publish (especially in newsletters or on external sites)

This category also tracks when visitors request to be unsubscribed from all e-mails from your site. This usually indicates the urgent need to improve e-mail marketing efforts and to be less spammy.

Economic activity

These graphs give a general overview of your site's economy. This includes points, banners, and eCommerce.

Feedback and engagement

These graphs give you an overview of the engagement between you / your site and your members. It also lists your top members based on views, forum posts, and points.

The Action Log graph in particular gives you a broad idea of how engaging your members are with the site and how many times they perform an action.

Forum

The forum category is better thought as a sub-category of Feedback and engagement. It showcases the activity within your forums: the number of new topics and posts created, and votes in topic polls.

Hits

Most of the graphs under the 'Hits' category only show hits from user agents that look like real humans (i.e. real humans using a web browser).

The 'Views per day of content existing' graph tallies and displays top content results based on average number of views per day to existing content. This is different from 'Content views' which tallies overall view counts to a piece of content.

The 'Unique views' graph only counts hits to the website once for any particular user, even if they have viewed multiple pages. A 'user' is defined by member ID for logged in users, or by IP address for non-logged-in users. As the web is inherently transactional, and we do not use tracking cookies in Composr, the unique measurement algorithm will never be completely imperfect. For example, a user may be on a connection that has changing IP addresses.

Inter-member engagement

These graphs are better thought as a sub-category of Feedback and engagement. They showcase how much your members are interacting with each other through Private Topics and Posts, and through sending each other points.

Punitive actions

These graphs give you an overview of the number of warnings issued over time and the frequency of punitive actions taken.

Referrers and referrals

There are various graphs that show referrers to your website, broken down in different ways. A referrer is a web page from another website that is linking to you.

Such data is invaluable in finding out what advertising is working (and conversely which advertising schemes aren't working), potential partners, the nature of your traffic, etc.

This category also includes invitations for when invites are enabled on your site.

Search traffic

These graphs give you an overview of what people are searching for (and how often) on your site.

Security

These graphs outline logged security issues such as hack attacks and failed logins.

Server performance

These graphs can give you an overview of how well your site is performing from a server perspective (e.g. page speeds, e-mails queued or sent). These graphs are very broad. For more detailed, thorough, and automatic checks, we recommend using the health_check addon (bundled by default).

Session behaviours

There are various graphs that will tell you about user sessions. These are telling you about the broader flow of users, and in particular how they enter and exit your website. This is important information to help you understand your users, and perhaps see what needs further work and what are strong points.

Performance

The load times and page speed graphs tell you about website performance. This is useful for:

1. Getting an overall idea of how fast your website is
2. Getting an idea what specific pages may be slow.

The times quoted are just server page generation time. It doesn't count any time taken transferring pages to the user, or for the user to download the page and page components (images etc).

This is useful information for optimising your site; finding bottlenecks and jams. For example you may find where things could be spread out more (e.g. putting fifteen blocks on different pages, instead of putting them all on the same one).

If web pages are taking seconds to just generate, you have a problem.

Filters

Most of the filters where you enter text for filtering support the ? and * wildcards.

Events

You can track arbitrary events of your choosing, and also some events are automatically tracked (such as members joining).
There is a graph that lets you see how many of each events happen, and also graphs showing the percentage rates of events happening for sessions, and the percentage rates of events happening for sessions originating from any tracking code.

Other software may use terms such as 'goals' or 'conversions': these are roughly equivalent to Composr 'events'.

There are a number of ways to record events happening:

• The inbuilt events are recorded automatically.
• You can use the {$LOG_STATS_EVENT,event} symbol to track an event from within your templates (i.e. when the template is shown, the event is recorded).
• You can use the JavaScript $cms.statsEventTrack function to track an event from within JavaScript code.
• You can use add data-click-stats-event-track="{ category: 'some event', nativeTracking: true }" on HTML <a> elements. This is actually a wrapper around the above.
• You can wrap URLs in links with the {$STATS_TRACK_URL,URL,event} symbol.
  E.g. <a href="{$STATS_TRACK_URL*,http://example.com/,Example.com Link clicked}">Click me</a>
  This handles the event recording within a redirection script, so does not depend on JavaScript. It also will track the external URL as having been viewed regardless of the event (the event parameter is actually optional).

Note that '3' and '4' also integrate with events within Google Analytics. Google Analytics integration is all about JavaScript code.

Here are the inbuilt events…

Event description	Event name	Event parameter	Event origin	Used with Google Analytics	Used with inbuilt stats system (by default)
Form filled in	Form	<formName>	Server-side	No	Yes
Purchase	eCommerce	<productName>	Server-side	No	Yes
Newsletter signup	Join newsletter		Depends	Yes	Yes
Click banner	Banner	<bannerCodename>	JavaScript	Yes	No (the banners module tracks clicks)
Click Comcode link	URL	<url>	JavaScript	Yes	No (In statistics, See Hits > Content views)
Click system links put out for many things (the HYPERLINK.tpl template)	URL	<url>	JavaScript	Yes	No (In statistics, See Hits > Content views, and Hits > Popular Pages)
Download a download	Download	<filename>	JavaScript	Yes	No (In statistics, See Feedback and Engagement > Downloads; also the downloads module tracks this)
Download an attachment	Attachment	<filename>	JavaScript	Yes	No
Play attachment audio	Audio	<url>	JavaScript	Yes	Yes
Play attachment video	Video	<url>	JavaScript	Yes	Yes
Open attachment overlay	Image	<filename>	JavaScript	Yes	No
Create support ticket	Create support ticket		JavaScript	Yes	No (In statistics, see Feedback and Engagement > Support Tickets)
Send contact message	<catalogueName>		JavaScript	Yes	No
Send contact message	Contact Us		JavaScript	Yes	No
Join	Join		Depends	Yes	Yes (also in statistics, see Conversions > Joining)
Do a social interaction (e.g. a like or a share)	social__facebook_like social__facebook social__twitter social__linkedin social__pinterest social__tumblr social__vkontakte social__sina_weibo social__qzone		JavaScript	Yes	Yes
Print the page	Print this		JavaScript	Yes	Yes
Bookmark the page	Add to favorites		JavaScript	Yes	Yes
Standard tracking via _t URL parameter	tracking__<parameterValue>		JavaScript	Yes	No
Get whitepaper from join block	Download whitepaper		Depends	Yes	Yes
Make a request to an API endpoint (/data/endpoint.php)	API Endpoint	<path>	Server-side	No	Sometimes (it depends on the API endpoint being called and how it is called)

Note there are some discrepancies:

1. To reduce noise within Composr's inbuilt stats, not every trivial event is logged.
2. For better reliability some events for Composr's inbuilt stats are recorded server-side while for Google Analytics they are activated in JavaScript (these are noted as 'Depends').
3. Some Composr inbuilt stats events are described as forms being filled in, while the Google Analytics events may be more specific (e.g. a support ticket being submitted). This is because a broader range of form events are captured for Composr's inbuilt events.

Tracking & Referrals

Tracking is useful for many things, including:

• Evaluating the success of marketing campaigns
  • On social media
  • In printed media
  • In newsletters
  • From partner websites
  • (and so on)
• Split testing different page versions to see which performs best (when combined with events)
• Rewarding users for referring new members to the website

Tracking codes are codes associated with a session that help you identify what caused the user behind that session to visit the site, or what split testing is associated with that user. Note that there may be multiple tracking codes associated with a session.

Composr has broadly integrated referral tracking integration:

1. The _t URL parameter may contain any tracking code you want to invent (*) and this is automatically registered by the stats addon when the URL is launched by a user.
2. The bundled recommend addon automatically includes tracking codes in the mailed URL for: the recommend module itself, the member doing the referral (if applicable), and the page being referred from (if applicable).
3. When a member joins, if they were referred by a member then that member will automatically be given points (points bundled addon) and made a friend (chat bundled addon).
4. The non-bundled referrals addon allows manually inputting tracking codes on joining and creates a system of referrer rewards.
5. The stats addon can draw graph of tracking codes, including measuring conversion rates against them.
6. The user lookup tool (in the bundled securitylogging addon) allows easily seeing if that user accessed via a tracking code.
7. Tracking is integrated with Tempcode. You can inject a tracking code of your choice in Tempcode by using the Tempcode {$SET_TRACKING_CODE,some code} or {$SPLIT_TEST,some code,some other code} symbol. You can use the {$HAS_TRACKING_CODE,some code} symbol to see if a tracking code is active for the user session.

* Tracking codes are structured as follows:

• Numeric values are for member IDs (referral schemes)
• Other values mean whatever you like and can be invented on whim as long as they are no longer than 80 characters (practically though, keep them very short because of comma-separation as described below)
• Multiple values may be passed together split by commas (for example, to track that a hit came because a member used the recommend module from a particular page)

Split testing

Split testing is a technique to try out different versions of pages and then use statistics to see which performs best. It is also (basically) known as A/B testing and multivariate testing.

You can perform split testing by combining tracking codes and events. The easiest way is to use Tempcode to:

1. Choose a random tracking code from a set and make it an active tracking code for the user session.
2. Generate different output based on the tracking code.

For example, your home page might contain:

You can then measure event success rates based on your tracking codes. Either inbuilt events, or new events you've created.

It's as simple as that!

When we are split testing tiny differences like the above we can be even more concise:

Even simpler!

KPIs

Most time-series graphs allow you to add that graph as a KPI. KPIs are special because:

1. They are available for quick reference from the staff checklist on the Admin Zone dashboard
2. You can set targets against the KPIs:
   1. You can see the targets on the graphs
   2. You can easily see an overview of which targets have and have not been hit
   3. Regular notifications will be sent to staff showing whether the targets have been hit or not

Spreadsheet export

Every graph has the option to download the data the graph is generated from, in spreadsheet form. This allows you to manually review it in detail, or use third-party graphing tools.

Implementation notes

Privacy

No default graphs or data tables provided by Composr statistics reveal personal information about users – everything is anonymous. This puts the system in line with GDPR (EU privacy legislation).

This happens because statistics graphs do not directly use data from the site to render the statistics. Instead, a scheduled task regularly takes site data and generates from it 'data buckets' which the statistics graphs then use. These 'data buckets' only contain data which is necessary to render the statistics and graphs; they do not contain personal information.

Pre-computed statistics

All statistics are generated from pre-computed data, which is itself based on raw data.
The pre-computation happens in the system scheduler, daily. The system scheduler is therefore required for Composr statistics.

Why do we do this? There are a number of reasons:

1. We can not hold onto raw data forever, due to GDPR (EU privacy legislation), yet we want to be able to see statistics going as far back as possible.
2. Some graphs require fairly complicated computations that should not be performed in real-time (for example doing geolocation of users, or tracing behaviour across user sessions).
3. We want to be able to process huge numbers of records without performance degradation.
4. Generally, we want statistics to be as snappy as possible, so that you feel comfortable dipping in and out of them, and setting different filters. If there's friction, you'll likely not spend as much time looking at your numbers as you should.
5. We want generated data to be readily available in other parts of Composr.

Limitations

One limitation to the pre-computation is that it does limit the amount of filtering possible to what we have built into our data structures. For example, you can set date ranges but only aligned to months.

Another limitation, to preserve site stability, is that statistics from over 31 days in the past will not be pre-processed. We have found sites will completely lock up even on performant dedicated servers (mainly because of the heavy use of database table locking in the process of pre-processing) if statistics have to pre-process multiple months of data. This means you could experience a gap in your data if any of the following occur:

• The system scheduler stops working for 31 or more consecutive days
• The statistics module repeatedly fails to pre-process data over the course of 31 or more consecutive days
• You manually delete existing statistical data from Admin Zone > Tools > Cleanup Tools, or from the database
• You use the site importer (imported content from over 31 days ago will not be factored in the statistics)
• You run the automated test suite for stats (testing_platform non-bundled addon)

Once statistics have been pre-processed, they will remain in the system unless manually cleared even if they are older than 31 days. It is only if statistics do not get pre-processed for over 31 days that they will not end up in the module.

Time zones

All times are calculated/shown in the configured website time zone, regardless of the time zone of visitors or the time zone your own account is configured to. This is a necessary limitation based on how statistics are pre-computed.
Some graphs allow showing hour-by-hour stats and also filtering by country, in which case you may get an idea for the local times by manually considering any local time zone differences.

Accessing other statistics

View counts

The statistics block

The statistics graph block