Solving Your Missing Email Problems

The War On Spam ! Are You a Casualty ?

Mailbox - Mail Client - So what's the difference ?
What is eMail Deliverabilty All About ?
Deliverability - Mail You Send Out
Receiveability - Mail Sent To You
Don't Forget Your Website Forms
Hmmmmmmm This is all a bit too technical
Frequently Asked Questions
• How does spam filtering work ?
• What is SPF (Sender Policy Framework) ?
• Is my router blacklisted ?
• How do I access my spam bucket ?
• Why did my mail go to spam ?

You simply cannot afford to live with a situation where you are sent mail that never arrives, and equally bad is mail you send out that never gets delivered. And neither you nor your correspondent - in either case - are aware of the non-delivery. It's a cost to both parties. And it happens. All the time. It's just that most of the time you are unaware. To some extent, it will be happening to you !

MAILBOX - MAIL CLIENT - SO WHAT'S THE DIFFERENCE ?

If you are having trouble receiving incoming mail, then it is critical that you understand this section.

It is important to understand that your incoming mail arrives at (and waits for collection from) a mailbox on a server located at your ISP's premises. So the term mailbox refers to the mail in that server which is usually quite remote from you - often in another country

Your PC will have a mail client loaded (such as MS Outlook, Thunderbird, MacMail, etc) which is a program that downloads the mail from your mailbox (before deleting it there) when you send/receive and then stores it on your PC where you can read it, store it in folders, reply, etc. So, the term mailbox refers to mail when at the server and mail client refers to mail when it is on your PC and no longer at the ISP's server mailbox.

- You use a webmail type browser URL to access the mail at the server, and especially to view the junkmail that has been filtered out.
- You use a mail client to administer the mail downloaded to your PC. - Your ISP will perform the primary junkmail filtering on your incoming mail and will send suspicious mail to junkmail at that level. So if you do not access that junkmail mailbox at regular intervals (recommended weekly) then you will never know that mail was sent to you.
- ISPs routinely remove old junkmail permanently - usually after three or four weeks!

WHAT IS EMAIL DELIVERABILITY ALL ABOUT ?

Some years back there was a flood of mindless destructive viruses and worms. They were pointless products of ego and filtering systems soon destroyed and defused most. But everything evolves and they were replaced with more sophisticated income earners such as malware, trackers, trojans, keyloggers and ransomware. Your ISP filtering is quite efficient in disinfecting these and moving them to junk folders. But the same principles are now being applied to the growing flood of incoming unsolicited advertising. The resultant spam filtering, especially where enthusiastically applied, can misidentify your legitimate incoming business email (known as a false-positive) and send it to the spam or junk folders. No bounce message will be generated and after a specific time (often 30 days) it will be automatically deleted. You and the sender could be (and often are) blissfully unaware.

But there is quite a bit you can do to fix this situation. Such as some once off domain wide tweaks, a change in your mail habits, and an annual check of your domain parameters.

DELIVERABILITY - MAIL YOU SEND OUT

Regarding your Domain :

Make sure your domain SPF Statement is correct and comprehensive
- this is by far the most common cause of false-positive spam problems
Check your domain reputation
Regularly check your router's public IP address for blacklisting
Use an online service to check sample emails for deliverability issues
Maybe get a DKIM key set on your domain (technically tricky).
Consider a mail delivery audit

Regarding User Habits :

Header : Your "From" (visible) and "ReplyTo" (hidden) mail addresses must match
Subject : Never leave it blank or use the "re:" prefix with nothing behind it
Subject : Never type in all capitals. Keep a mix with lowercase
Subject : Don't ever use the word "FREE" anywhere. Try "no charge" instead
Subject : Try not to use any punctuation
To : Use angled brackets around the mail address : ie <joe@doe.com> instead of just joe@doe.com
Content : Never start with "Dear .....", especially "Dear Friend" Just don't use "Dear..."
Content : Never type in all capitals. Keep a mix with lowercase
Content : Avoid using excessive punctuation in the body, eg. punctuation!!??
Content : Avoid substituting letters with numbers, eg. w3b h0sting
Content : Avoid using large font sizes and many different font colours in your email.
Content : Avoid having gaps between the letters in words, eg. l e t t e r s
Content : If you are only showing an image (ie a pic or poster) then make sure there are a few lines of text typed in as well.
Content : Avoid using gimmicky words or phrases that are commonly found in spam emails, eg: Click Here, Buy Now, As Seen On, Money Making, Get Paid, Make Cash, Pre-Approved, Hot Deal, Satisfaction Guaranteed, Lowest Prices, Save Big, etc.
Content :
Content : Avoid using shortened URLs (web links) in your email body, eg. bit.ly or goo.gl
Content : Avoid using rude language or poor spelling
Content : Avoid adding a large number of URLs (web links) to the email body

Also remember to :-
- Ensure that your internet IP address assigned to you by your Internet Srvice Provider is not listed on a spam blacklist.
- Add an accurate SPF and possibly a DKIM DNS record to your domain.

The Bottom Line for Email Deliverability:

When someone says they didn't receive your email then follow through. If it happens once it will repeat.
First step is to check that your router's external IP address has not been blacklisted.
Detailed instructions on how to do this are below under : Frequently Asked Questions / Is My Router Blacklisted ? (How Do I Know and What To Do)
The second step is to check your own outgoing emails for domain, security, reputation andcontent issues as any of these can cause problems. The easiest way is to use a free on-line service such as Mail-Tester which will give you a rating plus a detailed error report. Be sure to use a test mail that was similar (particularly in terms of content) to the one rejected by their server.
Ask them (the recipient) to check their spam buckets. The cause of the mail going in there (as recorded in the header) will probably be something you can change to fix the issue. Make sure they understand that they are looking for a spam bucket or junkmail folder in their upstream ISP mailbox and not in their PC client.
If the mail is there, then the recipient's mail server will normally put a note in the header which says exactly why it deemed it spam and sent it to junkmail. The answer is nearly always in the spam-score section of the email header in the junkmail folder. We can interpret this for you.
But if the mail is not there in the junkmail or spam folder, then this is a more serious issue than lost incoming mail as you cannot then rely on every mail getting to its destination, and there's nothing else to warn you (or the recipient) of that. Don't bank on the recipients checking their spam buckets as most people don't. And most people have auto mail receipts disabled.
Get hold of us (or someone) to start an investigation from source.

RECEIVEABILITY - MAIL SENT TO YOU

Make sure that every user knows how to access their upstream mailbox at their ISP and ensure that they do this at least once a week. They should understand the difference between the ISP mailbox and their mail client (such as Outlook).
The spam folders (sometimes called spam buckets or junkmail) should be checked and legitimate mail should be dragged to the inbox from where normal download will proceed.
Identify a staff member who can be trained to assist users with this function.
Consider lifting the spam score threshhold if there are many false-positives.
This could increase spam inflow but evaluate that on results.
Consider white-listing correspondent domains where a fix is dependant on them and may take time.
This could just be a temporary measure until fixed.

Best Practice :
Know how to examine the email headers of a spambucketed false-positive email to determine how the misidentification occured. All spam filtered emails that fail have the reasons appended into the mail header. Then you can figure out what corrective action to take. This helps the user, and others in the organisation, to receive mail timeously and minimises potential mail losses. Much of the time this will be caused by your correspondent's mail domains being malconfigured but let them know. Apart from their appreciation - that will mean their next mail to you should come straight through.

Also see the section at the bottom of this page :- How do I access my spambucket?

The Bottom Line for Email Receivability:

You need to check the spam bucket in your upstream mailbox at least once a week so that you don't lose incoming mail.
Then find out why that mail was spam filtered and make changes to stop that happening again.

DON'T FORGET YOUR WEBSITE FORMS

You've probably put in a lot of time and cost into preparing a website that brings in the visitors. It would be most unfortunate if all that effort was wasted because a hard won enquiry was lost simply because the form completed email was spambucketed.

Forms completed on a website are usually sent via server based mail servers and are subject to many different web variables from those that your web client sent mail encounter. So it is critical that these forms are tested regularly - and not just by filling one out and seeing if you receive it. The spam scoring may be near the threshhold positive score and the content that a user completes can be enough to tip it over the edge.

Best Practice :
We recommend that these forms should be tested with an on-line scorer and are cleaned and prepared to a near perfect level.
We use www.mail-tester.com who will allow you two free tests a day. Send us the results (or the testing address given to you) and we can interpret and advise as to what you need to do.

THIS IS ALL A BIT TOO TECHNICAL FOR US

PC Support people, to different degrees, should be able to deal with most of these issues. Your ISP and domain custodian (often the same) would assist.

You should start with a "Mail Deliverability Audit" which will measure the extent of the problem and provide a list of problems to fix. Your ISP may provide this service and if you are hosted with us we can obviously assist in this regard.

OTHER FREQUENTLY ASKED QUESTIONS

HOW DOES SPAM FILTERING WORK ?

When an email for you arrives at your ISP it is first virus scanned and then spam filtered. The spam checks are made against a sophisticated rule driven system that first mkes sure the mail is not whitelisted and then compiles the score, penalty item by penalty item. Your total score is then checked against the threshhold score for the domin. If it is lower than the threshhold then the mail is accepted. If it exceeds the threshhold then the spam score summary is appended to the email header and the email is placed in the spambucket and not in your inbox. After specified amount of time 30 days?) the spam identified email will be deleted.

WHAT IS SPF ? (Sender Policy Framework)

Sender Policy Framework is an anti-spam measure usually implemented by larger organisations. It expects you to have a statement attached to your domain's DNS record (an ISP function) which lists the different SMTP servers that your users send their mail to. This could include MWeb, Telkom, MTN, Vodacom, etc, etc. When their mail server receives a mail from you it checks that mail's headers to determine which was the first SMTP server that accepted your mail into the internet. It then looks up your domain's SPF statement to check that the SMTP server your mail went through is in the stated list. If it isn't, then an assumption is made that the sender address is forged and this is probably spam. Usually you would receive about 60% of the threshhold spam score for a major SPF failure.

Remember that some users may use multiple devices (desktop, laptop and phone) to send mail and each one could have a diffrent SMTP value. Everyone in the organisation sending mail has to have their SMTP address included in this statement.

IS MY ROUTER BLACKLISTED ? (How Do I Know and What To Do)

The mail header in your mail also lists yur PCs public IP address. Something like 196.30.22.55. This is the unique IP broadcast by your router and would be common to all the PCs inside your network. When your router is rebooted it loses this number and a new one is allocated by your ISP as the router logs on. Many (if not most) of these routers have been reported for spam activities (perpetrated, innocently or not, by a previous user) and now it's being used by you. Every one of your emails (organisation wide) are now carrying this compromised IP address. If the recipient's systems are filtering with blacklisting checklists then you could pick up heavily weighted spam score for this.

The only known remedy is to get your external public IP (just ask Google: "What is my IP") and check it with a service such as MXToolbox to ensure it is clean. If it isn't, then you need to reboot the router (wait 5 minutes before restarting) for another IP until you do get a clean one. If you are a big enough organisation you may want to get your own dedicated IP from your ISP - but this is not a cheap solution.

HOW DO I ACCESS MY SPAMBUCKET ?

Although the following help pages are for Hetzner/Xneelo hosted clients, the concepts are universal.

WHY DID MY MAIL GO TO SPAM ? (How Can I Stop It Happening)

Almost every mail server (but not GMail of course) will add a note to the header section of the email (which you ordinarily do not see) when it moves the suspect email to spam. This explanation can be read by revealing the raw headers. The procedure to view headers varies from webmail to webmail, and from mail client to mail client. The Microsoft Outlook method is to view the properties of the email when in your inbox . If using xneelo/Hetzner webmail, then you use the [More] > [Show Source] route.

Below is a typical spam score section from an xneelo/Hetzner header that details how the 9 points were accumulated. 5 is usually enough to get marked as spam. We can interpret these scores and help you fix as much as possible, but often it is your correspondent sending the mail to you that has the problem.

The difficult issue here is that we need to see the raw headers in your correspondent's copy of the email that they have fished out their junkmail folder. That's the only way we can see why that mail server would not deliver to their New Mail and consigned your email to their Junk Folder.

We suggest you send an email to your correspondent with the following wording.......

We have been having trouble sending email to you in that your spam filtering system on your email server has made a false-positive determination that our email is definitely spam and has consequently placed our email in the Junk folder.

Can we suggest the following remedy.

(A.) Contact your system administrator and request that our email address (your email) addy is "white listed for server delivery" at your mail server. This will ensure that your server will no longer filter mail from us to you.

(B.) Can you please take the trouble to retrieve one of our emails from your Junk folder, reveal its raw headers and send it back to us so that we can read the data that explains why it figured this was a spam mail. This helps us fix any mail deliverability errors on our side and we can also communicate back to you regarding any configuration issues on your side which may be effecting your mail receivabilty from us and other correspondents.
Revealing the headers (see Options > Source in Outlook) may require some help from your IT support.

Thank you for your cooperation.

Anything we left out, stuff you don't agree with.?
Good article, bad article.?
Please give us your comments and suggestions.