View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 4052 | Composr | core | public | 2020-01-26 22:55 | 2022-08-15 15:47 |
| Reporter | Chris Graham | Assigned To | Chris Graham | ||
| Priority | normal | Severity | feature | ||
| Status | resolved | Resolution | fixed | ||
| Summary | 4052: Logging framework | ||||
| Description | Rather than just logging to text files, make it configure on a per-log basis if logs should go to syslog. A sysadmin can then have the syslog be rsyslog, configured to send to a central network location. This is useful when hosting across multiple servers, not having to chase down logs on individual machines. For each log there'd be a choice whether to log locally and whether to log to syslog. Therefore any combination of either (or neither) may be configured. | ||||
| Tags | Type: Cloudification | ||||
| Attach Tags | |||||
| Time estimation (hours) | 6 | ||||
| Sponsorship open | |||||
|
|
To integrate with syslog you call openlog early in Composr init, set to LOG_USER - which allows you to set the 'ident', perhaps to the website domain. Then you call syslog when you do the actual logging. EDIT: Bad idea. This is relying on global state for the whole PHP process (I think anyway). We can just set a prefix directly in the syslog call. |
|
|
The logging framework should log with a number of standard parameters, including: 1) Date/time 2) Message components (an array) 3) Session ID of user 4) Member ID of user 5) IP address of user's client 6) IP address of server 7) Request URL 8) Request ID (something random generated at the start of a request, and the same code that would be used for the profiler filenames; also set via apache_setenv() so Apache can log it) It should do it in CSV format, so that it is easily parsable, openable in any spreadsheet program, so we can have a header row, and so we can have multi-line text in there. With a header row we can easily render out the logs in the admin_errorlog module to nice paginated tables. If logs are set to syndicate to syslog, do not show them in the admin_errorlog module as it would be showing the logs of one arbitrary server. Instead put out a message saying all log data should be read from syslog. The PHP error log may be a bit tricky, as PHP will write there in its own format in situation where our custom log handler can't kick in. So I suppose with the error log we'll have to continue to using PHP's format. Support more options for handling old log data. v11 will allow deletion of old log rows (for GDPR), but now we can also allow anonymisation and/or log rotation quite easily. |
|
|
It would be nice to use the same time format as Apache uses in the logs, at least on Linux. This makes it a lot easier to reconcile things. This works on Linux to get an exact match (and the first part is cachable)... $timezone = exec('date +%z'); $hours = intval(substr($timezone, 0, strlen($timezone) - 2)); $minutes = intval(substr($timezone, -2)); $date = strftime('%d/%b/%Y:%H:%M:%S', time() + $hours * 3600 + $minutes * 60) . ' ' . $timezone; Or just this... $date = shell_exec('date +"%d/%b/%Y:%H:%M:%S %z"'); I don't think it's reasonably achievable on Windows. And it assumes shell_exec is available. So we'd only do it if possible. |
|
|
Going over this now. Some changes... syslog will be supported. But also cloud logging will be doable via the database, synching new log lines across all server hosts. This is good for a small number of hosts, and will work almost out-of-the-box, as opposed to needing a lot of time and expertise to set up syslog nicely. Log settings will be configured on the first line of the logs themselves. TSV instead of CSV, as it is a lot easier to read for a casual user, and we can prohibit tabs from our data while could not do the same for commas. We will leave admin_errorlog. It is fine to see the log from one particular server and that will be clear as logs will now include server hostnames. And besides, the new cloud synching of log lines means that does not need to be the case. The user will know their settings so will know what to expect from admin_errorlog. |
|
|
" Support more options for handling old log data. v11 will allow deletion of old log rows (for GDPR), but now we can also allow anonymisation and/or log rotation quite easily." I don't want to do this, as I don't feel the extra time/complexity is worth it. If someone feels differently they should speak up. GDPR forces us to delete all log files, regardless of anything else. And, if users insist on logs being anonymised rather than just culled on a timer, then this would need doing to Apache etc, so us supporting some magic cleanup wouldn't really change anything. Log rotation should not be needed just because there is zero performance hit having large log files, as we can easily fseek deep into a log to read/write. That is apart from the disk space, which is cheap and plentiful nowadays, and which would still be used if log rotation was happening anyway. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2020-01-26 22:55 | Chris Graham | New Issue | |
| 2020-01-26 22:55 | Chris Graham | Tag Attached: Roadmap: v12 | |
| 2020-01-26 22:55 | Chris Graham | Relationship added | related to 3856 |
| 2020-01-26 22:55 | Chris Graham | Relationship added | related to 3147 |
| 2020-01-26 22:57 | Chris Graham | Note Added: 0006309 | |
| 2020-02-07 01:23 | Chris Graham | Note Added: 0006373 | |
| 2020-02-07 01:32 | Chris Graham | Note Edited: 0006373 | |
| 2020-02-07 01:34 | Chris Graham | Time estimation (hours) | 4 => 6 |
| 2020-02-07 02:15 | Chris Graham | Relationship added | related to 4103 |
| 2020-08-02 18:07 | Chris Graham | Note Added: 0006664 | |
| 2020-08-02 18:07 | Chris Graham | Note Edited: 0006664 | |
| 2020-11-07 20:11 | Chris Graham | Tag Attached: Type: Cloudification | |
| 2020-11-07 20:12 | Chris Graham | Relationship deleted | related to 3147 |
| 2021-03-15 17:24 | Chris Graham | Relationship added | related to 2652 |
| 2021-11-13 00:32 | Chris Graham | Note Edited: 0006309 | |
| 2021-11-13 03:53 | Chris Graham | Note Added: 0007165 | |
| 2021-11-13 04:11 | Chris Graham | Note Added: 0007166 | |
| 2021-11-17 21:28 | Chris Graham | Assigned To | => Chris Graham |
| 2021-11-17 21:28 | Chris Graham | Status | Not Assigned => Resolved |
| 2021-11-17 21:28 | Chris Graham | Resolution | open => fixed |
| 2022-08-15 15:47 | Chris Graham | Tag Detached: Roadmap: v12 |
