Very large sites are unlikely to use our inbuilt stats system, but rather something like Google Analytics - so it's not a big problem.

Patrick - look at sources/hooks/systems/tasks to see examples of current task hooks.

Tasks allow items to be queued for execution via Cron, not subject to timeouts, and run in an orderly manner so as to not over-stress the server.

A task is initiated using code like (code from cms_blogs.php)...

require_code('tasks');
return call_user_func_array__long_task(do_lang('IMPORT_WORDPRESS'), $this->title, 'import_wordpress', array($is_validated, $download_images, $to_own_account, $import_blog_comments, $import_to_blog, $import_wordpress_users));

Where the array is whatever options need to be passed to the task.

By default a task will run immediately. That's fine for testing them.

admin_stats.php is very old code and not ideal in how it is structured. You may choose to move a lot of it out to new functions that can serve data both for the task hook and for the admin_stats module.

Reassigned to me as I'm currently considering reimplementing the stats code for v11. The Health Check addon needs to be able to consume stats data, to be able to warn of concerning dips in stats - so changes beyond just merely moving code around is needed.

In v11 stats is being completely rewritten. There's now a pre-processing step, which happens in Cron. This means things like CSV download can happen in real-time as it's going from pre-processed data (much quicker to prepare a report).