Looking again now, the emails still appear to be sending.
So I don't know if there is a cut off point from the Fatal Error.

Yes, the mail queue is smart enough to auto-continue if this happens without issue. I will look into this issue soon.

Also I would like to ask what throttles the emails ?

I can see them going out in batches every 10 minutes - is it possible to reduce that time ?

Many thanks Chris

It is unthrottled in Composr, so probably you have CRON running every 10 minutes.

Doh - of course, I set it to that, when we had issues before and never changed it

Just looked and it is set to run every minute, so I had put it back.

The email queue is definitely sending the emails in batches with 10 minutes between them though

Ah, my mistake. It's not the mail queue, it's the newsletter drip sender. The newsletter doesn't run off the mail queue, it has it's own.

I have used hidden options in Commandr to adjust it.

:set_value('minutes_between_sends','1');
:set_value('mails_per_send','20');

(20 per 1 minute, instead of the default 60 per 10 minutes).

Composr was performing unnecessary Comcode conversion for HTML newsletters, then throwing away the result. I've fixed that. I've put it up to 100 every minute now. I've also fixed that inconsistent Composr startup times would make the gap 2 minutes, if the subsequent load time was slightly shorter than the prior load time (i.e. unsynchronised clocks).

Perfect thanks Chris.
I do appreciate how well you support us

Hi Chris,

So it sent about 10,000 emails, so that must of been the fatal error point.

Did you manage to find where the bug was ?

This leaves us in a bit of a dilemma, as we really do not want to send it out to everyone again, but want it to go to the rest that did not receive it. Is there a hidden way to start the send from a certain point, say if we know where it stopped ?

Cheers
Ade

:(

I think actually it was sending about half in each batch, before I fixed it. But that shouldn't even be possible because I fixed it too quickly for it to fail to send this many, given how slow it was sending. So I suspect I never fixed it (which I doubt), or the number reported is wrong?

How did you see this fatal error? Was it sent in an error email, when you manually ran CRON, in the error log?

Are you able to provide a list of the email addresses that it did send to, so I could potentially blacklist these in the code temporarily.

Hi Chris,

The error appeared in the window where you send the Newsletter, it came back with a green tick but at the bottom of the window it showed:

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 122880 bytes) in /home/vwgolfm/public_html/sources/tempcode.php(1690) : eval()'d code on line 207

I will attach a file with the emails of the people that have already had the email sent to them

Cheers
Ade

Oh! That was in a different part of the system to what I expected, I fixed an inefficiency but it seems not the bug.

I have made a few changes for you...

1) Disabled PHP memory limit during newsletter queue filling (where this issue actually happened)

2) Detected when an email uses no Tempcode, and avoid Tempcode computations in that case (vast improvement)

3) Added a block list feature, to help you re-send. Docs for this follow...

[title="2"]Blocking senders[/title]

If you wish to create a list of e-mail addresses to never send to, you can do so by making an [tt]uploads/website_specific/newsletter_blocked.csv[/tt] file. Put each e-mail address on it's own line. A CSV file with a single column is the same thing as a simple text file with one entry per line, so it's really simple to edit, however you wish to work. The filter works behind-the-scenes between you choosing to send out your newsletter, and Composr actually sending it.
The block list is useful for situations such as:
- A previous send failed part-way-through for some reason, and you have a list of users you know it did get sent to already.
- You maintain a manual list of users who have unsubscribed and you are very careful to not send to them, even if someone re-subscribes them.
- You want to block a list of e-mail addresses you know belong to competitors.
You may add additional columns to the CSV file, which will be ignored. You can use this for keeping notes, for example. If you do this make sure you save as a true CSV file (comma-separated) though, not a TSV file.

Cheers Chris !!

So if the csv exists Composr will use it, and there is nothing I need to add to the newsletter that I am sending out ?

Where do I upload the cvs to?
I might be being thick but I cannot follow your doc instruction.

Cheers
Ade

"So if the csv exists Composr will use it, and there is nothing I need to add to the newsletter that I am sending out ?"

Correct.

"Where do I upload the cvs to?"

uploads/website_specific/. This is a default Composr directory so should already exist on your Composr install.


I would test you have it working first by putting an admin email address in there and sending it to admins (for example), just confirming it only went to the non-blocked ones as you expected.

Then I'd put the true block list in and send as normal.


I have extended the system a bit for you just now. If you have the CSV in place you will see a message about it (see attached screenshot).

(Click to enlarge)

I have changed your CSV to the format we need and reuploaded with the correct filename, newsletter_blocked.csv. I have made this issue private to protect privacy on the addresses.

You should see this message (I just tested this CSV):
"You have a uploads/website_specific/newsletter_blocked.csv file in place, blocking 4,880 e-mail addresses from receiving the newsletter."

Thanks for this Chris.

Ok so I have uploaded the csv and added two of the admin email addresses

[email protected]
[email protected]

It says that it is blocking 4881 e-mail addresses (I assume that the rest have already opted out of the Newsletter ?)

I then sent a newsletter to the administrators group and I still got an email to both of these addresses :-(

Can you take a look please.

Cheers
Ade

"It says that it is blocking 4881 e-mail addresses (I assume that the rest have already opted out of the Newsletter ?)"

About 50% of the list supplied is duplicated. So if I open the CSV I see 9684 rows, but 4881 if I run a de-duplication.

I am looking into the block list apparently not working.

Please test again. It looks like I missed one line when manually transferring over the block functionality - the most critical one :S. Sorry about that.

I have tested it on my own machine.

Hi Chris, that works for me.

So I have kicked off another Newsletter send to the masses and on the Newsletter sending page I got another

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 122880 bytes) in /home/vwgolfm/public_html/sources/tempcode.php(1690) : eval()'d code on line 207

I thought that this limit was turned off now ?

Cheers
Ade

Oh, I'm sorry about this :(. I forgot to copy the line to disable the memory limit too. I've been a bit ill over the weekend and my concentration just isn't there. I've put that line in now.

The memory limit error again in general, I am still surprised, because my optimisation should have got rid of it. Unless there's Tempcode in the newsletter actually, which I suppose on reflection is fairly likely if you're using some of the default unsubscribe code which detects what kind of link to create.

Sorry to mess you about.

Sorry to hear that you are under the weather mate - thanks for fixing it.

Yes we have some tempcode for username and unsubscribe.

Get well soon !!

Cheers
Ade