View Issue Details

IDProjectCategoryView StatusLast Update
1563Composrcorepublic2024-07-25 21:55
ReporterChris Graham Assigned ToGuest  
PrioritynormalSeverityfeature 
Status newResolutionopen 
Summary1563: Change bounce reporting to automated tool / Remove stale users
DescriptionCurrently you can manually run a bounce scan. Change this to a "setup bounce detection" tool. It'll be very similar, but rather than allowing deletion it will just show the detected bounces in a results table, and save the settings into hidden options.

Then bounce detection will happen regularly via a CRON hook, updating the events table (3437) and facilitating the improved subscribers UI (2142).

Make sure we utilise our bounce API in mail2.php. The email_bounces table would probably be removed, as the events table would now serve this purpose. The full bounce email (subject and body) would be copied into the 'extra_data' field of the events table, to allow debugging / verification by a programmer.

Add new config options for automatic unsubscribing for bounced e-mail addresses if they happen more than n times across x days. ('x' and 'n' would be the config options). Include also an option of whether this means blanking out member e-mail addresses too.

Update tut_email after any changes made, so that the positive/negative spam-avoidance-advice correctly ties in with our debounce/destale functionality.
TagsRoadmap: Over the horizon, Type: Avoiding e-mail spamblocks, Type: Legal compliance / Privacy
Attach Tags
Time estimation (hours)6
Sponsorship open

Sponsor

Date Added Member Amount Sponsored

Relationships

related to 3437 Not AssignedGuest Newsletter events log 
related to 2142 Not AssignedGuest Improved subscriber management 
related to 603 Not AssignedGuest Newsletter tracking, include tracking code integration 
related to 3439 Not AssignedGuest Enhanced is_email_address function 

Activities

Chris Graham

2017-11-28 00:26

administrator   ~5279

All this has to be clearly documented.

Chris Graham

2022-10-31 01:25

administrator   ~7612

Last edited: 2022-10-31 01:34

This was a newsletter issue, but I think really it represents a broader need in Composr. Repeatedly sending emails to addresses known to bounce is likely to get us banned on particular email vendors (e.g. Apple, https://support.apple.com/en-us/HT204137).

We should have a Cron hook that scans for email bounces based on a set of rules specified in this issue. Bounces should then be passed to a new bounce hook type, which will unsubscribe from whatever that hook is dealing with. Hooks to implement:
1) Newsletter
2) Notifications
3) Mantis tracker

1- Simple removal of record
2- Explicitly turn off all notifications in the database (so they don't default back to on).
3- Turn off notifications in the preferences, which is this query: UPDATE mantis_user_pref_table SET email_on_new=0,email_on_assigned=0,email_on_feedback=0,email_on_resolved=0,email_on_closed=0,email_on_reopened=0,email_on_bugnote=0,email_on_status=0,email_on_priority=0 WHERE user_id=xxx; (these settings work as a filter AFTER whatever can trigger a notification, so they take precedence)

Chris Graham

2022-10-31 01:37

administrator   ~7613

Additionally we should not email users who have not logged in for a long time. See https://support.apple.com/en-us/HT204137 - "Periodically remove inactive subscribers from your list."

We should have a Cron hook that finds stale accounts, as defined by a config option specifying inactive timeout

This can also then call the bounce hooks, with a parameter specifying it's of a stale type rather than a bounce type.

Chris Graham

2022-10-31 01:38

administrator   ~7614

Last edited: 2022-10-31 01:38

And lastly, we should implement a button on the notifications screen to reset notifications to default settings. This allows a user to restore notifications to how they would normally be if they come back.

PDStig

2023-12-05 15:02

administrator   ~8098

A couple of additional feedback:

1) I think bounce detection should be integrated into the account registration process as well (if e-mail verification is on). Best practices are to keep your site from ending up on a blacklist. Unfortunately, some poorly-maintained webservers (including mine) get themselves on level 3 network-wide blacklists for failing to handle spammers. This might cause legitimate sites that happen to be on a blacklisted network to get blocked by e-mail providers (AT&T is especially strict about this). There are a couple ways we could go about this:
 - a) A registration status link. Upon registering, members are given a link with a unique / secure code to check the status of their registration at any time. This page could tell them when the site's attempt to send them an e-mail verification failed. It could also provide them status of accounts needing to be manually verified by staff.
 - b) If a user who just registered and is pending e-mail verification has an e-mail bounced, then we could perhaps have a config option on what to do: Do nothing, remove the e-mail address associated, delete the account, ban the account, or ban the account and IP address.

2) To be more compliant with the GDPR, we should separate the functionality of erasing the e-mail address / not e-mailing stale accounts into a different configuration, and allow for the ability to delete stale accounts automatically. This configuration allows us to pick what to do for accounts that have not logged in for x days:
 - Do nothing
 - Stop e-mailing the member
 - Stop e-mailing the member and unsubscribe from all newsletters and notifications
 - Delete the member's account
 - Delete the member's account and anonymise all data associated with the member
We should also have a config option allowing to specify an automatic reminder e-mail to members x days before their account is considered stale and will have this action performed.

Add Note

View Status
Note
Upload Files
Maximum size: 32,768 KiB

Attach files by dragging & dropping, selecting or pasting them.
You are not logged in You are not logged in. This means you will not get any e-mail notifications. And if you reply, we will not know for sure you are the original poster of the issue.

Issue History

Date Modified Username Field Change
2017-04-25 13:16 Chris Graham Tag Attached: Type: Avoiding e-mail spamblocks
2017-11-28 00:12 Chris Graham Relationship deleted related to 603
2017-11-28 00:17 Chris Graham Time estimation (hours) 4 => 6
2017-11-28 00:17 Chris Graham Summary Bounce reports => Change bounce reporting to automated tool
2017-11-28 00:17 Chris Graham Description Updated
2017-11-28 00:18 Chris Graham Description Updated
2017-11-28 00:23 Chris Graham Relationship added related to 3437
2017-11-28 00:23 Chris Graham Relationship added related to 2142
2017-11-28 00:23 Chris Graham Relationship added related to 603
2017-11-28 00:26 Chris Graham Note Added: 0005279
2022-10-31 01:19 Chris Graham Relationship added related to 3439
2022-10-31 01:20 Chris Graham Category newsletter => core
2022-10-31 01:25 Chris Graham Note Added: 0007612
2022-10-31 01:25 Chris Graham Tag Attached: Roadmap: v12
2022-10-31 01:34 Chris Graham Note Edited: 0007612
2022-10-31 01:37 Chris Graham Note Added: 0007613
2022-10-31 01:38 Chris Graham Note Added: 0007614
2022-10-31 01:38 Chris Graham Note Edited: 0007614
2022-10-31 01:44 Chris Graham Summary Change bounce reporting to automated tool => Change bounce reporting to automated tool / Remove stale users
2023-12-05 15:02 PDStig Note Added: 0008098
2024-03-26 00:58 PDStig Tag Renamed Roadmap: v12 => Roadmap: Over the horizon
2024-07-25 21:55 Chris Graham Tag Attached: Type: Legal compliance / Privacy