Troubleshooters.Com and T.C Linux Library Present

Emailing Out:
Nullmailer, Mutt, and Automated Email:
The Tiny Mail Transfer Agent

Copyright (C) 2007, 2011, 2013 by Steve Litt, All rights reserved. Material provided as-is, use at your own risk. 

Contents

Disclaimer

This page is an introduction to sending email without a full blown MTA -- not an authoritative text. This page may have errors -- in fact it probably does. DO NOT trust what you see on this page without verifying it for yourself. I am not responsible for any damage or injury caused by your use of this document, or caused by errors and/or omissions in this document. If that's not acceptable to you, you may not use this document. By using this document you are accepting this disclaimer.

The techniques in this document could be used to send spam. Don't do it. You could find yourself fined, prosecuted, and you'll certainly find yourself ostracised. If the people aren't your existing customers, or your use of email doesn't conform to laws including the CAN-SPAM act, don't do it.

Don't Panic, Here Comes the Cavalry

If you're trying to deal with Nullmailer and getting more frustrated by the minute, my Nullmailer Landmine Map is just what you need to get it running in style. For a more general overview, continue reading this page, but be aware you can always switch to the Nullmailer Landmine Map.

Rising to the SSL Challenge

Be aware that only very recent versions of Nullmailer support SSL. I *know* 1.11 does. I *think* 1.10 does. I doubt that the earlier ones do, although I could be wrong. As a practical matter, I'd recommend 1.11 or later, unless you have 1.10 installed. To see whether your Nullmailer supports SSL, do this:
libexec/nullmailer/smtp --help
Note that in the preceding, libexec is a subdirectory of some other directory, so you'll need to put in the whole path. If the preceding command lists an argument called --ssl, then the program is SSL enabled. Otherwise it isn't, and you must get a modern version and compile it with SSL enabled.

To compile in SSL, you need to do this:
./configure --enable-tls
make
sudo make install
sudo make install-root
When you do your make, it will stop each time it encounters an unfulfilled dependency. You use your package manager to install each of those dependencies. Here's a hint: Before you start, install the dev files for gnutls.

Introduction To Nullmailer

I'm pretty good at Linux, but configuring sendmail and postfix if far beyond my capabilities. For the most part it doesn't matter because I use either my ISP's or my web host's SMTP server, and I use Claws-Mail (and once in a while Thunderbird), both of which have built-in SMTP clients to transfer directly to those SMTP servers (MTA's).

But now orders for my Ebooks have gotten frequent enough that I'd prefer to create and email them with a process instead of manually. Trouble is, the mail executable requires an MTA. Sendmail and Postfix are out of the question -- they're way too complex, and if you do them wrong you've just become an open relay for the world's spammers.

So I use nullmailer.

Nullmailer is a tiny MTA, including a sendmail decoy that pretends it's sendmail. When you run the mail executable, it triggers the sendmail decoy, which calls the nullmailer-send program.

Nullmailer can be difficult. Very, very difficult. Error messages are maddeningly vague, like "Sending failed: Could not exec program" or "Could not load the config". Which program couldn't it exec? Why could it not load the config, and where is the config stored? It can appear as a black box. We just don't know.

Meanwhile, Internet documentation for Nullmailer is incomplete, to say the least. So is the documentation that comes with Nullmailer. Yes, I know, they tell you to "put your host in the defaulthost file", but please, WHICH host? The one for my computer? The one for my ISP? They don't say -- the just gleefully go on spouting docs so incomplete they're meaningless. Although this document isn't complete either, I hope it's complete when telling you what to put in which nullmailer variables, thus completing the rest of the docs for nullmailer. And remember, when things get really frustrating, you can go to my Nullmailer Landmine Map.

This is the Second Revision of this Document

The previous version of this document relied heavily on Kmail. I don't use Kmail anymore, and I couldn't possibly recommend it. Also, this version has an accompanying troubleshooting document called Nullmailer Landmine Map, which you can use to solve those really tough Nullmailer problems. This revision took place after much more Nullmailer experience.

Installing Nullmailer

You can install Nullmailer either as a package from your distribution or by from compiling from source, as documented in the HOWTO document that comes with the source code tarball, at least Nullmailer version 1.11. Personally, I'd do it from source code, because you'll have less black-boxy type stuff. This document describes a Nullmailer 1.11 installed by compiling source.

Be aware that if you want SSL, you need to start by making sure you have the gnutls library develop files installed so you have gnutls/gnutls.h, and then start your compile with this:
Please be aware that the preceding procedure is for version 1.11. For other versions, please see the doc files that come with it.

YOUR FIRST STEP -- Yes, I'm Shouting

I'm going to tell you a sad, tragic story of a week of lost wages. All because I didn't do this first step. I spent a couple days troubleshooting nullmailer, then, doing a diagnostic test, I accidentally erased everything in /etc and had to reinstall. Then it took me another three days to get nullmailer working. That whole time I told everyone who would listen that nullmailer was a no good black box, and I was finished with it. I got it working and have used it ever since.

MAKE SURE YOU HAVE A WORKING EMAIL SENDING PROTOCOL!  If you Thunderbird, use , use the one from Thunderbird. Likewise if you have Claws-Mail, Eudora, Evolution, or any other email client. Make sure you can send from one of those, then record the protocol. At a minimum you need the following info:
The last three of the preceding usually aren't used. The password usually the same password you use to retrieve your email (POP or IMAP).
NOTE

What I did, and I recommend you do too, is make a junk email account and use that for user and password. That way, if the password gets away from you, it's a junk account you can delete.


If you have SSL or StartTLS or TLS encryption, there is some documentation on the Internet about how to do that, but MAKE ABSOLUTELY SURE it works in that mode on your email client before trying it in Nullmailer. If in your email client tells you something about accepting an unknown certificate, that's probably going to stop you dead in Nullmailer -- that's what it did to me.

All this info should be available from your ISP, although it's usually given as a series of MS Outlook commands. My main point is, make sure you have all this working on Kmail or Eudora or Evolution or Mozilla Thunderbird or Balsa or Sylpheed or something like them, before you try it on Nullmailer, and once you try it on Nullmailer, make sure you copy it all exactly into Nullmailer's configuration.

Configuring Nullmailer

Configuration is usually done in the etc/nullmailer directory. Did you notice I didn't put a slash in front of etc? That's because sometimes it's off the root (the regular /etc directory), sometimes it's under /usr/local, and theoretically it could be almost anywhere.

On Ubuntu 11.04's package manager installed Nullmailer, and I'd assume other Nullmailer distributions, configuration is done in the /etc/nullmailer directory. When I compiled from source, it was the /usr/local/etc/nullmailer directory.

Anyway, wherever it is, there's one file, called remotes, that's always necessary, because it defines your ISP's SMTP mail server (the mail server to which you send your email). The remotes file has the following format:
smtpserver.your-isp.com smtp --port=<portno> --user=<loginname> --pass=<yourpassword>

These were all the info you collected in the preceding article. If your authorization type is "Login", you might need a --login-auth argument also.

You're probably going to also need to configure a file called me and one called defaulthost. Each will probably need to contain your ISP's domain name, e.g. your-isp.com.

Nullmailer Mechanics: A Simplified View

My nullmailer came with six executables:
Here is the Mental Model of a standard Nullmailer system:

Mental Model diagram of Nullmailer

Nullmailer is based on a mail queue in which newly created emails are placed, and from which they are sent via the libexec/nullmailer/smtp executable (unless a different executable is used). New emails are placed in the queue by nullmailer-inject, via nullmailer-queue, both of which take the basic email file and tweak it a little bit, adding other headers. I'm not sure, but I think other executables like mail and sendmail use nullmailer-inject to queue up messages. Note that things like the real SMTP Sendmail and Postfix are mutually exclusive with Nullmailer on any single machine. On a given machine, if you have a full blown SMTP server, you can't have Nullmailer, and vice versa, so Nullmailer replaces the sendmail executable with one of its own, which I believe calls nullmailer-inject.

This mutual exclusivity creates a problem where you need to use a full blown SMTP server outside the given computer in order to deliver local email. If you use your ISP's SMTP server, this can be a security risk. I'm pretty sure there's a way to modify the sendmail executable supplied by Nullmailer to enable local delivery, but I don't currently know exactly how to do it, and it's beyond the scope of this document.

Anyway, the sending part is done by nullmailer-send, a continuously running program that every once in a while sends some more queued messages. Also, when nullmailer-inject places a new message, nullmailer-queue wakes up nullmailer-send , via the pipe at var/nullmailer/trigger, so the new message can be sent immediatly, if the queue is close to empty.

As a practical matter, on a desktop computer the queue will always be almost empty except in the following circumstances:
  1. You just ran software to place a hundred or so new messages via nullmailer-inject.
  2. Many of your queued messages are in error so they're never sent and deleted.
  3. Your machine has several users sending lots of email.
The nullmailer-queue program places messages submitted by nullmailer-inject into the queue at (var/nullmailer/queue) and informs nullmailer-send, via the trigger pipe, to scan the queue. The nullmailer-send program then submits each queued message to libexec/nullmailer/smtp to do the real work of submitting the email to your ISP's Mail Transfer Agent (like Sendmail or Postfix or Qmail). When the smtp executable reports success back to nullmailer-send for each message, nullmailer-send deletes the successfully sent message from the queue so it doesn't get sent again.

The mailq program lists the emails still in the queue, making it a valuable troubleshooting tool, although those messages can still be seen by looking in the Nullmailer queue directory.

Special Directories

The directory structure will depend on how your Nullmailer was installed. Linux distribution packages tend to put its executables in /usr/bin and /usr/sbin, while 1.11 compiled on the target machine puts them in /usr/local/bin and /usr/local/sbin.

Nullmailer uses some special directories. One excellent way to find them all (and all other files related to Nullmailer) is to do something like this:
locate -i nullmail > nullmailer.files
Be sure to update the tree for the locate command before doing so. On my computer, you update it with sudo updatedb.

One is its configuration directory where it keeps its configuration files.

Configuration Directory

Once again, exact location depends on installation. Config files from Linux distribution packages tend to be in /etc/nullmailer, while from a compiled tarball they tend to be in /usr/local/etc/nullmailer. Note that even this is up for debate -- they can be put anywhere by tweaking arguments to the ./configure command.

WARNING
Thinking your config files are in one place when they should really be in another is a frequent cause of the dreaded  "Could not load the config" error message, while having your executables somewhere where you don't expect them is a frequent cause of the "Sending failed: Could not exec program" error message.

My Nullmailer Landmine Map document shows you how to work with such ambiguous error messages in order to fix your problems.


On my computer the configuration directory is /usr/local/etc/nullmailer. Yours may be elsewhere. The one file this directory must contain on every installation is remotes, which tells Nullmailer which SMTP server to send the email to, and how to interact with that SMTP server. There are several other possible config files:
File
 Used by
 Use
remotes
 nullmailer-send
 Defines the SMTP server(s) to which you send email, and the protocol with which to access them.
me
 nullmailer-inject Badly documented, but my experiments tell me its best to put your ISP's domain name here.
defaulthost nullmailer-inject Overrides the hostname in /etc/mailhost. As a practical matter, it's often necessary to set this to the domain name of your ISP, e.g. earthlink.net.
defaultdomain nullmailer-inject If the hostname determined by the defaulthost file or /etc/mailname has no dots, then this is bolted on at the back of that hostname.
pausetime nullmailer-send From the man page:  The  number  of  seconds  to pause between successive queue runs when there are messages in the queue (defaults to 60).  If  this              is  set  to 0, nullmailer-send will exit immediately after going through the queue once (one-shot mode).

In other words, how many seconds will you wait to let messages accumulate before sending them.
adminaddr nullmailer-queue The email address to whom Nullmailer sends failures and errors.
helohost
 nullmailer-send More protocol stuff. Beyond the scope of this document.
sendtimeout
 nullmailer-send The man page is unclear on this, but it appears to be a timeout on an individual message send and appears to default to one hour. One hour is probably an excellent value on dialup, but on a properly functioning high speed (10Mbps) Internet I can't imagine a realistic send from a home computer lasting more than ten minutes, and if the file is that big, you're probably going to run into limits at the SMTP server.

Queue Directory

The queue directory is where nullmailer-inject puts emails to be sent, and where nullmailer-send finds its emails to send, and then deletes the sent emails. Once again, the locate command is your friend. On my Ubuntu 11.04 computer with package-installed Nullmailer it was /var/spool/nullmailer/queue. On my Xubuntu 12.10 computer with manually compiled Nullmailer, they're in /usr/local/var/nullmailer/queue/.

In the queue directory you can watch the comings and goings of files, look at files, change files, and any other diagnostic tests you can think of. Or, you can use the mailq executable to view queued messages.

Testing and Troubleshooting

NOTE

If you really get in a troubleshooting pickle, go to my Nullmailer Landmine Map document for some detailed, practical help.

On Ubuntu 11.04, the normal way to run nullmailer-send was with the following command:
service nullmailer start
On my self-compiled nullmailer-send version 1.11 on Xubuntu 12.10, I do it like this:
nullmailer-send
or, if you want it to start when you boot your computer, you can make it a service in /etc/init.d or make it a Daemontools service (see


The preceding command runs it as a daemon, which isn't what you want while testing and troubleshooting. You want it running in the foreground so you can see messages realtime. To do that, as user root run it like this:
nullmailer-send
Here's a nice script that can send a test message on a Linux computer, obviously after you set variables for names and email addresses in the top section:
=================================
#!/bin/bash

# START OF USER CHANGABLE VARIABLES
sender=yourself@yourdomain.com     ##### MUST BE YOUR EMAIL AT THE SMTP SERVER
youremail=$sender                  ##### FEEL FREE TO MAKE IT SOMETHING ELSE
                                   #####   IF YOU HAVE A SECOND EMAIL ADDRESS
friendemail=friend@friendsdomain.com

yourname="Yourfirst Yourlast"
friendname="Friendfirst Friendlast"
#  END  OF USER CHANGABLE VARIABLES


emailfile=myemail.email
dtime=`date`
ltlt="<"
gtgt=">"

echo "Subject: Nullmailer test at $dtime" > $emailfile
echo "From: $yourname  $ltlt$sender$gtgt" >> $emailfile
echo "To: $yourname  $ltlt$youremail$gtgt" >> $emailfile
echo "Cc: $friendname $ltlt$friendemail$gtgt" >> $emailfile
echo "" >> $emailfile
echo "Sent at $dtime" >> $emailfile
echo "" >> $emailfile
echo "$yourname was here" >> $emailfile
echo "and now is gone" >> $emailfile
echo "but left his name" >> $emailfile
echo "to carry on." >> $emailfile
echo "" >> $emailfile
echo "This is a second paragraph thats kinda long, really really long, so long that I truly hope that it does the right thing and wraps." >> $emailfile
echo "" >> $emailfile
echo "Sincerely" >> $emailfile
echo "$yourname" >> $emailfile

cat $emailfile | nullmailer-inject -h
=================================

If you look at myemail.email you can see what the raw file looks like, before nullmailer-inject passes it off to nullmailer-queue, which does its rfc822 magic on it.

Notice the -h argument for nullmailer-inject. That means use only the addresses in the file header, and nothing on the command line argument. I did that so all the addresses are in one place, the file, which makes things less confusing.
Dufus Alert

If any of your recipients is a mailing list, make sure your sender address is a member of that mailing list, or else your message won't go through. Sounds obvious, right? Well, as the possessor of multiple email addresses, I got nailed by this obvious dufus move and lost a half hour trying to figure it out.

Security Issues

Nullmailer has one security issue that would make it dangerous on a system with several users: it passes the email password from the remotes file to the libexec/nullmailer/smtp executable as a command line argument, so anybody can see it using the ps command or in places in the /proc tree. True, for small messages, the smtp executable runs for less than a second, but still, this doesn't give me a warm and fuzzy feeling, and would be downright unacceptable if other people used this computer.

This doesn't have to be a show-stopper. If this bothers you too much, you can modify your remotes file and the libexec/nullmailer/smtp code as follows:
Another security issue is that unless I built it wrong, 1.11 as it comes from Untroubled.Org runs nullmail-send and libexec/nullmailer/smtp as root. I don't know how safe or dangerous this is, but a lot of distributions run it as user nullmail. You could probably do that too. If you do, and you make the password transfer modification I mentioned earlier, you'd need to have that password file readable by user nullmail.

Personally, being the only person who uses this machine, and having a good firewall, I'm not overly concerned about either of these security issues, but if you are, you can fix them.

Getting it to Work With Mutt

You might find that Mutt doesn't work with Nullmailer.

You know why Mutt doesn't work with nullmailer? Because Mutt uses the environment variable $EMAIL as the sender address, and this address is typically a local address instead of something at your ISP or anything else on the Internet under control of the global DNS system. So the SMTP server drops the message on the floor.

Here's a script you can use to get Mutt to work with Nullmailer:
=========================================
#!/bin/bash
oldemail=$EMAIL
EMAIL=your-email@your-isp.com
export EMAIL
mutt
=========================================
The preceding makes sure that Mutt's "From" header is your-email@your-isp.com, not something on your local box.


Back to Troubleshooters.Com * Back to Linux Library