View Issue Details

IDProjectCategoryView StatusLast Update
5063Composrcore_forum_driverspublic2022-11-21 14:09
ReporterPDStig Assigned ToGuest  
PrioritynormalSeverityfeature 
Status newResolutionopen 
Summary5063: Use get_table_count_approx for forum drivers / Do not count bots created by phpBB towards number of members
DescriptionCNS uses get_table_count_approx for counting members/topics/posts, because InnoDB is very slow. Ideally forum drivers should use this to.

Also:
phpBB creates a bunch of bot accounts for web crawlers. Do not count these towards total site members in Composr when using the phpBB driver.
TagsNo tags attached.
Attach Tags
Time estimation (hours)2
Sponsorship open

Sponsor

Date Added Member Amount Sponsored

Activities

Chris Graham

2022-11-21 03:17

administrator   ~7670

From our POV it is inefficient to try and work out what are bots, as those no direct way to do it - we'd have to check group membership, meaning a join and less efficient SQL counting mechanism.

I wonder how phpBB itself handles this. Does it not count bots? Does it show a total at all?

Regardless, this isn't a big issue, we just wanted to raise it on the tracker in case people care and to have the discussion.

PDStig

2022-11-21 03:26

administrator   ~7672

Last edited: 2022-11-21 03:28

It is possible to get the member count with 2 queries: first, grab the bot group ID, second, run a COUNT on the users table for all users whose primary group does not match that bot group ID (primary group is on the users table, so no JOIN is needed)

Chris Graham

2022-11-21 08:20

administrator   ~7677

Is this going to be reliable or does it hard-code a group name that might be edited / vary by language?

Also in the ideal world we'd be using get_table_count_approx for the member/topic/post count queries in the forum drivers, as counting full table rows on InnoDB is very slow. We're currently only doing that for CNS. If we do that, we either need to not do the filtering or actually have a third query for the number of bots to subtract (rather than doing it as part of a WHERE clause). I'll add that to this issue.

PDStig

2022-11-21 14:09

administrator   ~7680

It looks like it would be reliable; the group name is BOTS and cannot be edited in the phpBB interface. I ran a Spanish install and it was still BOTS in the database (the other group names were also the same).

Add Note

View Status
Note
Upload Files
Maximum size: 32,768 KiB

Attach files by dragging & dropping, selecting or pasting them.
You are not logged in You are not logged in. This means you will not get any e-mail notifications. And if you reply, we will not know for sure you are the original poster of the issue.

Issue History

Date Modified Username Field Change
2022-11-19 18:11 PDStig New Issue
2022-11-21 03:15 Chris Graham Time estimation (hours) => 1
2022-11-21 03:16 Chris Graham Category General => core_forum_drivers
2022-11-21 03:17 Chris Graham Note Added: 0007670
2022-11-21 03:26 PDStig Note Added: 0007672
2022-11-21 03:28 PDStig Note Edited: 0007672
2022-11-21 08:20 Chris Graham Note Added: 0007677
2022-11-21 08:21 Chris Graham Summary Do not count bots created by phpBB towards number of members => Use get_table_count_approx for forum drivers / Do not count bots created by phpBB towards number of members
2022-11-21 08:21 Chris Graham Description Updated
2022-11-21 08:21 Chris Graham Time estimation (hours) 1 => 2
2022-11-21 14:09 PDStig Note Added: 0007680