Composr Tutorial: Relaying Errors and Statistics to Composr

Written by Patrick Schmalstig
Composr CMS has an optional and secure way to automatically relay some errors you encounter on your site to the core developers (via the homesite); we call this the telemetry service. The telemetry service can also relay some basic statistics about your site so that developers can improve on the software according to how it is used.

While optional, we highly recommend enabling telemetry at least for relayed errors. That way, core developers can be informed of potential software bugs you might miss or neglect to report normally to the tracker. With that being said, this also should not serve as a replacement to reporting issues; not every error gets reported (e.g. critical errors which bail out Composr do not get reported).

The telemetry service is also fully GDPR-compliant. And you can opt-out of it (and even request automatic deletion of your data) at any time.


Configuring telemetry

Image

Telemetry options on the installer (step 2)

Telemetry options on the installer (step 2)

(Click to enlarge)

Image

Telemetry options under Privacy / legal compliance options

Telemetry options under Privacy / legal compliance options

(Click to enlarge)

There are two places you can set your telemetry options. The installer will prompt you on step 2 when you agree to the licence whether you want to relay errors and/or statistics to the developers.

You can also change your telemetry options at any time under Admin Zone > Setup > Configuration > Privacy / legal compliance options.

Viewing relayed errors

Image

A relayed error from low-level logging

A relayed error from low-level logging

(Click to enlarge)

Image

A relayed error from data_custom/errorlog.php

A relayed error from data_custom/errorlog.php

(Click to enlarge)

Image

The relayed error status page; the error has not yet been resolved

The relayed error status page; the error has not yet been resolved

(Click to enlarge)

Image

The relayed error now has a message from the developers

The relayed error now has a message from the developers

(Click to enlarge)

When an error occurs and you have telemetry set to relay errors to the core developers, the error might (but not always) get relayed. Errors which are very likely not to be bugs, and critical errors which bail out the software, will not be relayed. It is important that you report any errors you encounter that you believe to be a software bug to the tracker so the developers are aware of them.

You can check whether an error was relayed in one of two ways. You can either look under Admin Zone > Audit > Low-level logging, and observe the "Relayed?" column in the error log. Or, you can view data_custom/errorlog.php manually, where relayed errors will have a line under them like this: TELEMETRY <GUID>.

You can then view the status and potential information about that relayed error by clicking the link ("Yes" items under the "Relayed?" column will be hyperlinks), or by going to homesiteurl/telemetry/<GUID>.htm where <GUID> is the GUID you pulled from data_custom/errorlog.php.

Common errors, or errors which already have a resolution, will be marked auto-resolved by the homesite and will have some instructions / troubleshooting for you to follow. Otherwise, the status will be 'open', meaning a core developer has not reviewed the error yet. Check back to the page often (and consider adding the error to the tracker) for updates.

Viewing telemetry status

Image

Viewing the status of a site with the telemetry service

Viewing the status of a site with the telemetry service

(Click to enlarge)


When telemetry is enabled, you can view a basic overview of your site's status with the telemetry system under Admin Zone > Audit > Telemetry status.

You will see what the telemetry service knows about your site, such as its URL and name, software version, when the telemetry service last checked if you still had Composr installed, the last time your site sent statistics, how many errors were relayed total, and the members / hits last reported (if applicable).

Deleting your data

If you want the homesite telemetry service to delete all telemetry data associated with your site, you must follow this process; we do not accept requests through e-mail or other means as this is the only way telemetry can verify you own the site you want erased:
  1. Disable telemetry under Admin Zone > Setup > Configuration > Privacy / legal compliance options.
  2. Open data/installed.php in a text editor.
  3. Un-comment the indicated lines in the file which makes it output 'Remove me!'.
  4. Wait at least a few days for the telemetry service to pick up on the file (telemetry only scans data/installed.php once per day).

The page at Admin Zone > Audit > Telemetry status will let you know (via a message at the top) when your site is still registered in the telemetry service. The message disappears when you are not registered anymore.

Understanding telemetry security (advanced)

Telemetry data, particularly relayed errors, can be sensitive in nature as they may contain full server paths and passwords. For this reason, Composr encrypts the data sent to the telemetry service on the homesite for the core developers. And the homesite uses several processes to verify the integrity of the data sent. Anything sent which does not conform to these standards will be rejected with an error.

Encryption is done using cryptographic keys located both in data/keys and data_custom/keys. These are generated using PHP's libsodium library. Keys specifically for telemetry will be prefixed telemetry-*.

There are several keys used in telemetry:
  • data/keys/telemetry.pub
    This is the public key for the software distributed to anyone who downloads it. It is specific to the major.minor version of Composr you download. While the public key is freely distributed and available, the private key is not and is only located on the homesite.

    This key is used to encrypt anything sent to the telemetry service. The homesite will then decrypt the data using the relevant version's private key. This verifies the data was sent from a site running Composr and verifies its version number.

    The core developers can invalidate these keys at any time simply by deleting the private key off of the homesite and releasing a new minor version with a new public key should they suspect a severe security breach with the telemetry system.
  • data_custom/keys/telemetry-site.key, data_custom/keys/telemetry-site.pub, data_custom/keys/telemetry-site-sign.key, and data_custom/keys/telemetry-site-sign.pub
    These are the public and private key-pairs specific for your site. They are generated when you enable telemetry and are destroyed when you disable it. The sign ones are used to sign and verify that data sent from your site actually came from your site. The non-sign ones are used to encrypt telemetry data sent by your site.

    When you enable telemetry, your public key and public sign key is sent to the homesite (encrypted by the software key), and the homesite remembers it for when your site sends encrypted telemetry data. That way, it can decrypt your data.

Let's run through the process:
  1. Enabling telemetry
    • When you enable telemetry, your site will generate data_custom/keys/telemetry-site.key, data_custom/keys/telemetry-site.pub, data_custom/keys/telemetry-site-sign.key, and data_custom/keys/telemetry-site-sign.pub keys. It will also generate a random "challenge" string and save it to data_custom/telemetry_challenge.txt.
    • Then, it will inform the homesite telemetry service that you enabled telemetry by sending the public (.pub) keys and the challenge to the homesite, encrypted by the software public key (data/keys/telemetry.pub).
    • The homesite then decrypts the payload. The homesite will verify the authenticity of the registration request (to prevent malicious actors from registering sites on their behalf) by calling your site's data/installed.php script, which will return the contents of data_custom/telemetry_challenge.txt and then delete the file. It will then compare the returned challenge with the challenge that was sent in your payload. If they do not match, the registration request is rejected.
    • The homesite then stores your public keys. Note your private keys (.key) remain private on your server, cannot be accessed, and are not transmitted (so even the homesite does not know them).
  2. Sending telemetry data
    • When your site sends telemetry data, it will first encrypt the data using a key-pair of your site's private key (data_custom/keys/telemetry-site.key) and the software's public key (data/keys/telemetry.pub).
    • Then, your site will sign the data with the private sign key (data_custom/keys/telemetry-site-sign.key).
    • This payload is then sent to the homesite.
    • The homesite locates your stored public keys according to your base URL.
    • Then, it verifies the signature of the payload using your public sign key.
    • If it's valid, it will then use a key-pair of your public key and the software's private key (according to the version you are running) to decrypt the payload and read/process the data.
  3. Number 2 also happens when you go to Admin Zone > Audit > Telemetry status (but just for some arbitrary data) to prove to the homesite you are who you say you are when you request to see your status.


See also


Feedback

Please rate this tutorial:

Have a suggestion? Report an issue on the tracker.