Troubleshooters.Com Presents

Linux Productivity Magazine

February 2012

Escape From Kmail

Copyright (C) 2010 by Steve Litt. All rights reserved. Materials from guest authors copyrighted by them and licensed for perpetual use to Linux Productivity Magazine. All rights reserved to the copyright holder, except for items specifically marked otherwise (certain free software source code, GNU/GPL, etc.). All material herein provided "As-Is". User assumes all risk and responsibility for any outcome.

Career skills nobody else teaches:
Books
Courseware

[ Troubleshooters.Com | Back Issues |Troubleshooting Professional Magazine ]
Don't upgrade to Ubuntu 11.10 before first deciding exactly what to do about Kmail!  --  Steve Litt.

CONTENTS

Editor's Desk

By Steve Litt
In November 2011 I installed Ubuntu 11.10 on my laptop in anticipation of a business trip, copied my Ubuntu 11.04 desktop's Kmail file hierarchy to the laptop, and ran Kmail on the laptop. Kmail announced it was now called Kmail2, with a new storage mechanism called Akonadi, wouldn't work with the old mailboxes anymore, and would I like to use its conversion feature to convert my old mailboxes to Kmail2, or would I rather just rebuild the mailbox hierarchy from scratch? My momma didn't raise no fool, I didn't want to transfer hundreds of folders one by one. I chose "convert".

The conversion bombed in the middle. No problem, I'll restore the kmail directories from backup, and convert again. Oops, this time Kmail2 told me it would only convert once, and I'd have to do it manually.

Ummm, no. So many mailboxes, so little time.

A little reading told me Kmail2 required some of its data to be in a database called Akonadi. So let me get this straight. All these years the one reassurance I've had was that my old emails were in standard file formats, either mbox or maildir. Now, to continue on with Kmail2, I'd need to put some email data into Akonadi, learn how to back it up, and maybe learn how to tweak its data. That's just not going to happen.

I Googled for others encountering the one-time-only Krashing Kmail Konversion, found several, even some claiming to have a way around the one-time-only thing, but they didn't work.

Ubuntu 11.04 is getting old. I need to upgrade to 11.10. But I can't make that upgrade until I get my email data out of Kmail and into something else. I was staring vendor lock-in right in the face, and it was scary. Suddenly replacing Kmail had become my top technical priority. And I had not the slightest idea how to do it.

The remainder of this document tells the story of how I escaped from Kmail, both my experiences and the architecture and configuration used to do it. What this means is although this conversion took me about a week, you can use the information in this document to do it much more quickly -- maybe in less than a day.

If you've been using Kmail and have become concerned that Kmail is going the wrong direction, kick back and enjoy this issue of Linux Productivity Magazine.
Steve Litt is the author of Twenty Eight Tales of Troubleshooting.   Steve can be reached at his email address.

Joe User's View of Email

By Steve Litt
Ten years ago, I'd viewed the Email constellation pretty much like Joe User: Input my ISP's SMTP server address into Kmail's "Send" facility, input my ISP's POP3 server address into Kmail's "Receive" facility, and bang, I was in business. Of course in actuality I knew it was more complex because there were port numbers to worry about, and authentication protocols, but basically I just gave Kmail my ISP's information, and Kmail did the rest.

A few years later, in order to implement Spamassassin on my desktop, I changed it so Fetchmail grabbed my ISP's POP3 and called Procmail to deposit the mail in file /var/spool/mail/slitt, and Kmail would then get incoming mail from that file. I got this setup from friends' advice and off the Internet, and understood not a bit of it.

Over the years, within Kmail, I added filter after filter, mailbox after mailbox, to accommodate my various mailing list memberships and other special mail. I also kept old email segregated into a tree called "oldfolders", because, for instance, if you had 20,000 emails in your inbox, Kmail would run incredibly slowly. Some of these mailboxes were in the mbox format, and some in maildir. Some old ones were even in a bastardized mbox format from Eudora, the email client I'd used while running Windows before March 2001. After all, what is commonly called "the mbox format" actually encompasses four differing formats. By last month I had well over 100,000 emails stored in these various maildirs and mboxes.

Can you see why when I felt helpless when I was forced to leave Kmail? I knew almost nothing about the architecture of email, and had over the years built up a Kmail system almost impossible to import into another client.
Steve Litt is the author of Thriving in Tough Times.   Steve can be reached at his email address.

My First Escape Attempt

By Steve Litt
Actually, my first try at leaving Kmail had been in 2005. At the time my main reason was simply that I don't like KDE. I know a lot of people using KDE daily who never have problems, but my experience on both Mandrake/Mandriva and Ubuntu is that KDE stands for Krash, Delay, Expand. It crashed regularly, with that irritating crash handler asking me for details, like I have time to do that twice a day. With KDE apps, especially Kmail, my experience was that things would slow down more and more and more. Eventually a top command would show an instance of dbus-daemon using 99% of CPU, and I'd have to kill it. Things got so bad that I created a cron-driven program that would kill any dbus-daemon instance that stayed above 94% for more than ten seconds. Yes, I know non-KDE apps use dbus-daemon too, but it always seemed like KDE apps were the ones to make dbus-daemon go crazy. KDE-launched stuff would Expand into both RAM and disk space.

And then there's the irritation factor. To use KDE apps I needed be conversant in extraneous stuff like Nepomuk and Akonadi. I mean really, I just wanted to send and receive email -- is this necessary? And just to make sure it's irritating, they give them those names. Akonadi sounds like fifth graders singing "na, na -- na nah, nah". And Nepomuk sounds like the gleeful utterances of six year olds splashing through a swamp.

In 2005 I tried to switch over to an email client called Sylpheed. Sylpheed was close but no cigar -- it had some "features" even more irritating than crashes and runaway dbus-daemon instances. I bit the bullet and stuck with Kmail, making a mental note that if I needed to migrate away from Kmail in the future, Sylpheed was a good candidate.
Steve Litt is the author of the Rapid Learning for the 21st Century. Steve can be reached at his email address.

Email Architecture from 40,000 Feet

By Steve Litt
Here's how email works...

The sender's MUA hands off to the MSA which sends to the MTA which sends to another MTA which calls an LDA to deposit in the email file. Then the recipient's MRA uses POP3 to pull the message onto his computer, and hands off to his LDA, which deposits the message in his email file. Then the recipient makes his MUA pull the message off the file and into the MUA, where the recipient can read it. If the recipient decides to reply to the message, the recipient becomes the sender and the whole thing starts again.

Confused? Yeah, so am I. Let's try a graphical approach. Pretend you sent me an email yesterday, and you use the mutt email client. It would look something like this:

Typical email between individuals
In the preceding, the boxes are processes, the boxes with curved sides are files, and the lines between boxes are communications or interfaces. The black communications and interfaces are part of the send process, whereas the heavier blue ones are part of the receive process. But for the time, let's discuss the preceding diagram.

MUA or Email Client

MUA stands for "Mail User Agent". It's the part the user manipulates. Another name for it is "email client". Some common GUI email clients include Kmail, Thunderbird, Sylpheed, Claws-Mail, Balsa, Eudora, and (ugh) Outlook. Common text-only email clients include pine, elm, and the extremely well respected mutt.

MSA or SMTP On-Ramp

I think MSA stands for "Mail Submission Agent". That's very confusing, because email gets sent a lot of different ways. That's why I prefer the phrase "SMTP on-ramp". That makes its function crystal clear.

Once you've composed your email on your MUA, the MSA must get it into the SMTP (Simple Mail Transport Protocol) system by sending it, using SMTP protocols, to an SMTP server, typically the one at your ISP. Most email clients (MUAs) have a built-in SMTP on-ramp, so you'd just configure that SMTP server into your email client, and your email client's built in SMTP on-ramp sends it. But with Mutt, and the mail executable, and maybe others, you need a separate and distinct SMTP on-ramp, so in those cases you'd use Nullmailer, or other SMTP on-ramps such as sSMTP or several others. For a long list of SMTP on-ramps (often called "nullmailers"), see http://linuxmafia.com/faq/Mail/nullmailers.html.

MTA or SMTP Server

MTA stands for "Mail Transport Agent". Often they're simply called "SMTP servers." Examples include Sendmail, Postfix, Exim and Qmail. The SMTP on-ramp, whether part of the MUA or standalone, sends the email to your ISP's SMTP server. That SMTP server sends it to my ISP's SMTP server. An SMTP server can do exactly three things:
  1. Receive an email
  2. Relay an email to an SMTP server on a different computer
  3. Call an LDA (Local Delivery Agent) to deposit the email in a file on the local computer.
The MTA changes the email's headers during those three operations.

So the mail gets passed from SMTP server to SMTP server until it reaches the computer on which it is to reside, and then the SMTP server hands off to an LDA to deposit it as a file.

MTAs (SMTP servers) are incredibly difficult to configure and keep secure. It's incredibly difficult to prevent their use by spammers, thus incurring the dreaded blacklist. This is why I've always chosen not to employ an SMTP server on my local machine. In my migration away from Kmail, one requirement was that I needn't set up an SMTP server.

LDA, MDA, LMDA

LDA, MDA, LMDA, they all mean the same thing -- a program that writes an email to a file. The file could be an mbox file, or a file in a maildir tree, or a file in an MH tree. The acronyms stand for Local Delivery Agent, or Mail Delivery Agent, or Local Mail Delivery Agent. For the rest of this article I'll use the acronym LDA.

The oldest and most recognized LDA is procmail. Others include maildrop, deliver, postdrop, and others. Also, various MTAs (SMTP servers) have their own LDA's, such as Sendmail's binmail, Postfix's postfix-maildrop, Courier's courier-maildrop, and Dovecot's builtin LDA. My setup uses Procmail because it's simple, robust, and time tested.

MRA or Mail Retrieval Agent

MRA stands for Mail Retrieval Agent. It goes out to a remote computer, and using either POP3 or IMAP, grabs waiting emails from that remote computer and brings them to the local computer. Once again, see the diagram:
Typical email between individuals

In the preceding diagram, the mail you sent is already in the mailbox file at Steve's ISP. Steve's MRA, which is called fetchmail, every few minutes queries his ISP's mailbox file for email destined for him, and if there is any, fetchmail brings it home. Now the only question is, what does fetchmail do with this email?

The traditional answer to that question is, submit it to the local SMTP server. But the truth is, it can also submit it to an LDA, as in the following .procmailrc command:
fetchmail -d300 -m /usr/bin/procmail -d %T
The -d300 means repeat every 300 seconds or 5 minutes, and the -m option passes the mail to the command /usr/bin/procmail -d %T, which deposits as files or parts of files the email as per the rules of ~/.procmailrc.

So, in the preceding diagram, Fetchmail circles every 300 seconds to grab messages from my ISP's mailbox file, and for each message calls Procmail to put it in my local mailbox file on my computer. Some time later, I read that mailbox file into Kmail's own mailbox structure using Kmail.

Multifunction Components

Here's the diagram one more time:
Typical email between individuals
Many products incorporate several of the preceding functionalities. For instance, almost every email client has a built-in SMTP on-ramp, thereby eliminating the need for Nullmailer and the Single Mail File at the top left of the diagram. Almost all email clients incorporate builtin POP3 functionality and built in mailboxes within the email client, thus eliminating the need for Fetchmail, Procmail and the mailbox file at the bottom left of the diagram. So if you and I were both using Kmail, configured to directly hit our ISP's SMTP and POP3 servers respectively, the diagram would degenerate to the following:
"Kmail Does Everything" Architecture
Diagram of personal email architecture using Kmail configured for ISP


A lot simpler, isn't it. But, as you'll see later in this document, knowledge of all the other components is essential to being able to make your email setup do what you want.
Steve Litt is the author of the Universal Troubleshooting Process Courseware. Steve can be reached at his email address.

Emergency: First Steps

As mentioned, a notebook conversion to Ubuntu 11.10 frightcasted the hideousness of Kmail2, convincing me that a quick and deliberate escape from Kmail was needed in the next couple months, so that I could upgrade my main box to Ubuntu 11.10 in a reasonable timeframe. A little web research showed that in the years since 2005, Sylpheed had been joined by other, similar email clients such as Claws-Mail (a fork of Sylpheed), Thunderbird, Balsa, GNUmail, Mullberry, Spicebird, and Zimbra.

NOTE

Actually, Claws-Mail began as the Sylpheed-Claws fork of Sylpheed in 2001, but I didn't know about it in 2005. If I had, perhaps I'd have converted in 2005 and saved myself a lot of grief later on.

Better yet, Barry Fishman of GoLUG told me about MH, a series of CLI programs that could be combined to create an email client, as well as scripted email tasks. That's what I wanted, so I started working toward converting to MH.

Well, MH has no converter to convert mbox or maildir to its native mh format. Kmail doesn't have such a converter either, and even if it did, I didn't want to do the conversion directory by directory. So I decide to write myself a folder-recursive converter, looked on the Internet, found some Python code, got it running, and Whalla!, it produced 100GB of mailboxes where there had been only 5GB. And there was no reasonable way to section it down to the root cause.

Meanwhile, a week or so before that, at a GoLUG "meeting after the meeting", somebody had suggested I "use a local IMAP server to import from Kmail, translate, and export to an email client that stores its mailboxes in MH format". At the time I understood nothing of what he said, but filed the concept away for safe keeping. Hearing what the first guy said, another GoLUGger said if I got it into a local IMAP server, I might as well simply keep it on the IMAP server and simply look at it with an IMAP aware client. At the time this sounded like a bad idea because I then could not use the MH utilities, but I also stored that idea away in my head.
Steve Litt is the author of the The Key to Everyday Excellence. Steve can be reached at his email address.

IMAP Basics

By Steve Litt
Here comes the most collossally oversimplified definition in history: IMAP is a way that your email client can view email files still on the server.

If you know enough to laugh at the preceding sentence, please suspend disbelief for a few minutes -- this document will later elaborate to make it not so oversimplifed. And if you don't know enough to laugh at the sentence, that's perfect, because you'll understand my exact intent in doing what I did to escape from Kmail.

Another thing. IMAP is not only a technology, but a lot of people consider it a philosophy. "I would never use POP3. I only use IMAP! That way I can hit my email with any computer anywhere, and not lose anything. And if I see email I don't want, I can just delete it without downloading it."

The old Cloud philosophy -- keep your mail on the server, and let the vendor control the technology. I see three things wrong with that:
  1. Eventually your emails will exceed the vendor's limit, and then what do you do?
  2. You're likely to be more careful in backing up and taking care of your emails than they would.
  3. Do you really want your most personal email -- contracts, tax documents, legal matters, love letters -- do you really want those on the Internet where somebody can get them?
So when I write about an IMAP server on this web page, I'm writing about IMAP server software called Dovecot running right on my desktop computer. You know how it gets from my ISP to my local Dovecot server? Good old POP3.

The only IMAP server I'm familiar with is Dovecot, so that's the one discussed here. What Dovecot does is take a tree of maildir or mbox directories, and serve them out via the IMAP protocol, typically at one of these ports:

No security
 143
Secure IMAP  over SSL
 993

Sometimes IMAP is set to other ports, but the preceding two are typical.

IMAP has an online and offline mode. In the offline mode it stores copies of all your emails on the local machine so you can work without a TCP connection. That's great if your ISP hosts the IMAP server and you sometimes do email where there's no Wifi. But if you have your own local IMAP server on your desktop or laptop computer, all it does is double the disk space consumed by your email. If you're anything like me, you have many years of accumulated email, and many such emails have attachments. The last thing you need is a double set of emails. Use online mode all the time if you're using your own on-computer IMAP server.
Steve Litt is the author of the Troubleshooting: Just the Facts. Steve can be reached at is email address.

The Pivotal Suggestion

By Steve Litt
Things were bad. All my attempts to write maildir to MH and mbox to MH scripts had failed miserably, with no reasonable way to troubleshoot them. It looked like I'd need to go the IMAP route, but I had absolutely no idea how to set up an IMAP server, because I knew absolutely nothing about IMAP except you could read your mail while it was still on the server. I emailed every one of the 8 LUG mailing lists to which  I belong, begging for ideas to transition my emails from Kmail to a different email client. Most of the suggestions involved setting up my own IMAP server, and I had absolutely no clue what IMAP even looked like.

Then a TriLUG member suggested I use a dummy Gmail account's IMAP to do the transfer. I responded saying while it was a good idea, many of my folders had info too private to go into the cloud, and promptly forgot about it.

Then the next day it hit me that I could use Gmail as a Rapid Learning Proof of Concept (POC -- see my Rapid Learning books). I set up my Gmail account, and managed to get Claws-Mail to see my Gmail IMAP account. I transferred emails, folders, and just had my way with it. Then I did the same with Thunderbird and Kmail. Now that I understood what IMAP was and what IMAP did, and how to configure a client to interact with IMAP, setting up my local Dovecot became quite doable.

The TriLUG member's suggestion was the turning point. From that point on, I worked off todo lists, knowing the exact route I'd take to Kmail independence.
Steve Litt is the author of the Quit Joblessness: Start Your Own Business. Steve can be reached at his email address

Learning IMAP from Gmail

By Steve Litt
If you're like I was and don't have a clue what IMAP really looks like in action, your first step is to get yourself a Gmail account and experiment with it. Note that in this article, you'll make a Gmail SMTP account as part of the  natural way of setting things up, but won't really use it. All sent mail will be from your existing, known-good email client and ISP SMTP account.

Set up your Gmail account on Gmail

So, if you don't yet have a Gmail account, do this:
  1. Go to http://gmail.com. It will redirect.
  2. Click the "Create an account" square in the upper left.
  3. Fill in the information, remembering your username and password.
  4. After you've put in all your info, test it by going to http://gmail.com, putting your username (with @gmail.com) in the username field, the password you just made up and put in on the password field, and then click the "sign in" square. For the remainder of this exercise, assume your Gmail login is myacct@gmail.com and your Gmail password for that login is mypass.
  5. Observe the screen you see. Going down the left of your screen is a list of mailboxes and mailbox views. Most of the rest of the screen is a list of messages. Just above this list is a list of formats like Classic,Important first, Unread first, Starred first, Priority Inbox.
  6. From a known-good email client with a known-good SMTP, send an email to the new account, myacct@gmail.com. Once it gets done sending, go back to your Gmail web page.
  7. Click "Inbox" at the top of the mailbox column, and click the "Classic" format above the message list, and you should see the email you just sent yourself. Don't click on it, or that will register it as read. If you don't see it at all, investigate.
If you see the new email message, you're done testing Gmail.

Use Thunderbird to interface with your new Gmail account

Next you need to connect to it with Thunderbird's IMAP. Do the following *within Thunderbird*.
  1. File->New->Mail_Account
  2. Fill in the first screen
    1. Your Name->Text of your full name, or what you want to be known as
    2. Email address->myacct@gmail.com : The Gmail account you just created in Gmail
    3. Password->mypass : The password for that account
  3. The "Mail Account Setup" screen appears, automatically testing until it comes up with the right settings for Gmail IMAP and Gmail SMTP. Make sure the IMAP radio button is checked. When the calculations are complete, click the Create Account button.
    1. NOTE: The default settings above give this account the SMTP for your Gmail account, which will send mail out as myacct@gmail.com. If that's not how your want your email going out (possibly because you don't want to change all your mailing lists), you can do it later. DO NOT click the Manual Setup button at this point. In spite of the aforementioned, keep in mind this is basically just a temporary test, so don't worry about emailing from this new address at this point.
  4. In the folder list on Thunderbird's left side, look for and open myacct@gmail.com or whatever you called it.
  5. Make sure you see the email message you sent earlier. If not, troubleshoot.

Create Gmail IMAP folders in Thunderbird

In this exercise you'll create some folders and subfolders, and move email between them. All the work is to be done in Thunderbird.
  1. In Thunderbird's left side account/folder list, highlight the myacct@gmail.com entry.
  2. In the upper left is a tab with myacct@gmail.com written on it, assuming you did the preceding step. Just below the legend myacct@gmail.comon the tab is a legend that says something like "All Folders" or "Unread Folders" or "Favorite Folders" or "Recent Folders" or "Unified Folders" or something like that. To the right  of that legend are two small arrows, one pointing to the left, one pointing to the right. These arrows alternate between"All Folders" or "Unread Folders" or "Favorite Folders" or "Recent Folders" and "Unified Folders". You MUST, MUST, MUST operate the arrows to make sure the legend says "All Folders". That is the ONLY way you can be confident that your folder operations will work in a logical way.
  3. Right click that entry, choose "New Folder", and type mysubfolder into it. Make sure the "Create as subfolder of" dropdown is set as myacct@gmail.com. It most likely already will be.
  4. Note that a folder called mysubfolder appears below myacct@gmail.com.
  5. Right click myacct@gmail.com again, choose "New Folder", and type secondsubfolder into it. Make sure the "Create as subfolder of" dropdown is set as myacct@gmail.com. It most likely already will be.
  6. Click mysubfolder and then right click it, select "New Folder", and create one called mysubsubfolder. This time, make sure that "Create as subfolder of" is mysubfolder, which it should be.
  7. Note that now a little expand triangle appears to the left of mysubfolder. Click it and see mysubsubfolder.
  8. Click the Inbox folder, click the test message you sent earlier.
  9. Right click that message, then select "Copy to", then myacct@gmail.com, then mysubfolder, then mysubsubfolder.
  10. Navigate to the mysubsubfolder and verify that a copy of the message now exists there.
  11. Look at the Gmail webmail page and verify that you now have folders mysubfolder and secondsubfolder, and that mysubfolder has mysubsubfolder inside it, and mysubsubfolder has the message inside it.
Sit back and contemplate what just happened. You just used Thunderbird to fly-by-wire control Gmail's IMAP server, located in Oregon, Georgia, Virginia, North Carolina, South Carolina, or who knows where else -- Google is very secretive about its hardware. I want you to think of it as fly-by-wire. Sure, "front end" or "client" means about the same thing, but "fly-by-wire" does a better job conveying the fact that all the data, all the information, is on the server.

Today's commercial pilot operates controls in the cockpit, and those controls send electronic orders to  the engines, landing gear, pressure system and the like. Gone are the days when there was actual physical linkage between cockpit and engine, landing gear and pressure system. The reason today's commercial airliner system is called fly-by-wire is those electronic commands go over a wire. In the same way, Thunderbird is your cockpit, and the Gmail IMAP server is the engine, landing gear and pressure system.

Move Gmail IMAP folders in Thunderbird

Here's where the fly-by-wire system breaks -- moving and deleting folders. If you do enough moving and deleting, sooner or later something will fail -- you'll be unable to make a move, or your system will get into a wierd state. When that happens, try to read the error message. Sometimes it will tell you that you already have a folder by the same name in your Trash Can. That's easy -- File->Empty_Trash.

If that doesn't do it, you'll need to go onto your Gmail webmail web page, click "Manage Labels" in the folder list (you may need to click the "more" link to see "Manage Labels", and change the parent of a folder, go back to Thunderbird, double click the account in the folder list, doubleclick it again, and you'll see the change. This would be the equivalent of trying and failing to put the landing gear down from the cockpit, and going into the belly of the plane with a wrench to do it directly. Luckily, email isn't as safety critical as an airplane.

My experience so far with IMAP and various email clients is that with all email clients, you can get yourself into state where the email doesn't match the IMAP server -- the cockpit doesn't match the landing gear. Luckily it doesn't happen very often. Luckily, you can often fix the problem with a different email client, which may have its own problems but doesn't have this one. And more luckily still, you always have the option of taking a wrench to the IMAP server itself, whether using Gmail's webmail, or manipulating Dovecot's Maildir directory hierarchy on your local computer.

Now, let's have a short exercise in moving folders. It may fail -- big deal. Try it.
  1. Drag mysubsubfolder from mysubfolder to secondsubfolder. Note that it works (we hope).
  2. Go into Gmail's webmail page, and verify that mysubsubfolder is really under secondsubfolder now.
  3. Drag mysubsubfolder from secondsubfolder to mysubfolder. Note that it works (we hope).
  4. Go into Gmail's webmail page, and verify that mysubsubfolder is really under mysubfolder now.
You've just done fly-by-wire work on Gmail IMAP, through Thunderbird. We used Thunderbird not because it's necessarily the best, but because it's the easiest and quickest to hook up to an IMAP server.
Steve Litt is the author of the Rules of the Happiness Highway. Steve can be reached at his email address.

Thunderbird Settings For Gmail

By Steve Litt
Look at these Thunderbird settings for Gmail. Thunderbird set them up for you, but other email clients aren't so nice, so it pays to look at and understand them.

Thunderbird Gmail Settings
GENERAL SETTINGS

Account Name
myacct@gmail.com (Note -- can be any text)
Your Name
Your Name, or how you want to be known
Email Address
myacct@gmail.com
Reply-to Address
Blank
Organization
Blank
Signature text
Blank
Use HTML
Unchecked
Attach Signature from file instead
Unchecked
Attach my vCard to messages
Unchecked
Outgoing Server (SMTP)
Leave empty, or use your normal non-Gmail account



SERVER SETTINGS

Server Type
IMAP Mail Server (settable only when first created)
Server name
imap.gmail.com
Port
993
User Name
myacct@gmail.com
Connection Security
SSL/TLS
Authentication method
Normal Password
Check for new messages at startup
Checked
Check for new messages every
Unchecked
When I delete a message
Move it to this folder: Trash
Clean up (expunge) inbox on Exit
Unchecked
Empty Trash on Exit
Unchecked
Local Directory
Leave at default value



COPIES AND FOLDERS

Place a copy in
Checked, choice is "Other" Folder on: Sent Mail
Cc these email addresses
Unchecked
Bcc these email addresses
Unchecked
Keep message drafts in:
"Drafts" folder on: myacct@gmail.com
Keep message archives in:
Other: All Mail
Keep message templates in:
Templates folder on myacct@gmail.com
Show confirmation dialog when messages are saved
Unchecked



SYNCHRONIZATION & STORAGE
Note: This section is vitally important to prevent
your local computer from getting clogged up
with copies of your IMAP folders!
Keep messages for this account on this computer
Unchecked (vital that it is unchecked!!!)
Or else you get copies on your local machine.
Synchronize the most recent

1 days
Don't download messages larger than
Checked, 1KB
Don't delete any messages
Checked



SECURITY
This section will be covered later in this document.

Please remember, Thunderbird has the easiest, most "we do it all for you" IMAP connectivity, so it makes an ideal learning tool. Make sure you look at and understand the settings.
Steve Litt is the author of the Thriving in Tough Times. Steve can be reached at his email address.

Connect Other Email Clients to Gmail IMAP

By Steve Litt
Now that you have somewhat of an understanding of IMAP via Gmail, try setting up other email clients to work with your new Gmail account:
You probably wonder why I listed Kmail, when my whole motivation was to leave Kmail behind. The reason is that IMAP is your escape hatch from Kmail. You'll transfer your Kmail folders to IMAP, and then access them from your final client.

Play with all of these until you can get them communicating with Gmail IMAP. Don't be sad if you can't get GNUmail to work -- I couldn't either. I also had some big problems with Sylpheed, and at first it was very hard to get Kmail working with IMAP, although obviously that's a must.

Once you have a good feel for getting arbitrary clients communicating with an IMAP server, you'll be in a much better position to set up and use your own local IMAP server.
Steve Litt is the author of the Manager's Guide to Technical Troubleshooting. Steve can be reached at his email address

Dovecot

By Steve Litt
Dovecot is a free software IMAP server. I installed it with Ubuntu 11.04's package manager. On Ubuntu 11.04 it's configured by editing file /etc/dovecot/dovecot.conf. In order to get it working, for simplicity without SSL or TLS, here are the lines you must insert or alter in the existing (stock) /etc/dovecot/dovecot.conf:


CONFIG LINE
MEANING
protocols = imap imaps
Make it serve out IMAP, which is the whole purpose.
disable_plaintext_auth = yes
Disable plain text authentication, UNLESS the login is from this computer, which in this application it is. "yes" is more secure than "no".
log_timestamp = "%Y-%m-%d %H:%M:%S "
Not necessary, but I like sortable times.
mail_location = maildir:~/mail/Maildir:INBOX=~/mail/Maildir/.INBOX
I wanted my tree head at~/mail/Maildir and my inbox as .INBOX below that.
INSERT FOLLOWING NAMESPACE

namespace private {

separator = /
??? Maybe should have been ".", I don't know. But slash works. Slash here DOES NOT cause subfolders to be actual subdirectories. The Dovecot code would need modification to do that.
prefix =
This is the default namespace.
inbox = yes
This (default) namespace has the inbox
}
End of (default) namespace

Configured as preceding, your email client should be able to access it at port 143 at 127.0.0.1, by inputting your computer password.

Steve Litt is the author of Twenty Eight Tales of Troubleshooting.   Steve can be reached at his email address.

Procmail

By Steve Litt
In the good old days of Kmail, all my filtering was done by Kmail itself. That was cool, because everything I did was with Kmail. But with my new setup, all email is stored in my local IMAP server, and can be hit with any client. It would be silly to set up filters in each client I use. So instead, I have Procmail do the filtering. In my new setup, Procmail is, in the words of Joe Biden, a big deal.

To refresh your memory, Procmail is a Local Delivery Agent (LDA), that puts email files in the proper directory after being called to do so by an MTA such as Sendmail or Qmail, or, in my case, a Mail Retrieval Agent such as Fetchmail. Every five minutes, my Fetchmail wakes up, grabs all the email from my ISP's POP3 server, and hands it off to Procmail, who formats each email as an Maildir-formatted file and puts it in the proper directory. Procmail is automatically aware of how to handle Maildir, as long as, in $HOME/.procmailrc, the destination has a slash at its end. For instance, the following is a Maildir specific destination:
.oldfolders.in.2009/
And here's a non-Maildir (mbox) specification:
/var/spool/mail/slitt
Note that the Maildir destination ends with a trailing slash, but the mbox destination doesn't. Basically, when procmail sees the trailing slash, it puts the new directory below the destination, so that whatever is accessing the Maildir can process it and then move it to the cur directory that's a sibling of the new directory.

Procmail's actions are governed by its configuration file, which is $HOME/.procmailrc. In this file, like most Unix files, everything after a pound sign (#) is a comment, and you can use the pound sign to comment out lines.

.procmailrc Environment Variables

The top part of $HOME/.procmailrc. sets special environment variables and also environment variables you might need. Here's the top of mine:
DEFAULT=$HOME/mail/Maildir/.INBOX/
MAILDIR=$HOME/mail/Maildir/
LOCKFILE=$HOME/mail/.lock
VERBOSE=no
LOGFILE=$HOME/procmail/log
GARBAGE=/dev/null
SUPREMUM=9876543210    #PROCMAIL SUPREMUM NUMBER,
                       #SEE http://www.perlcode.org/tutorials/procmail/proctut/proctip2.pod
In the preceding, I set up my Maildir base to $HOME/mail/Maildir, and set the default place to put mail in the main inbox, DEFAULT=$HOME/mail/Maildir/.INBOX/. I set up the logfile to be outside of my Maildir tree. The last two environment variables aren't used by Procmail itself, but are used by my procmail recipes later on in $HOME/.procmailrc. The $GARBAGE is set to /dev/null, but if I need to do some debugging, I can change it to a specific folder so I can see what would normally vanish. $SUPREMUM is set to a high enough number to trigger short-circuit logic in the recipes, as will be explained later in this article.

.procmailrc Recipes

What Procmail does with each email is dependent on a set of recipes that follow the environment variable part of the $HOME/.procmailrc file. A simple Procmail recipe (and all of mine are simple) consists of the following three sections:
  1. The recipe header. It begins with a colon followed by zero, and signifies the beginning of a new recipe. If this recipe will operate a lock file, it has another colon right after the zero.  There are flags that can be placed after the zero, but that's way beyond the scope of this document.
  2. The regular expression. This line starts with an asterisk, a space, and then the regular expression. If  something in the email matches the regular expression, this recipe is applied. There can be multiple regular expression lines, in which case, under most conditions, they are anded, so both must match. There's a way to or them, and that will be discussed later in this article.
  3. The destination. On simple systems like mine, this is typically a filename. Or, if it ends in a slash, it's a Maildir directory whose new subdirectory will receive the file. The destination can also be a script. This will be discussed later in this article.

Recipe Header

The recipe header typically looks like this:
:0
Or, if you want to invoke a lock, which is a very good idea even with Maildir, add a trailing colon:
:0:

Regular Expression

Regular expression lines all start with an an asterisk and a space. Here are some typical ones:
* ^Subject.*\[MyLinuxGroup\]
The preceding matches all emails whose subjects contain [MyLinuxGroup] in the subject line. The carat at the beginning of the regex means that the S in the word "Subject" is the very first letter of the line, and in emails that defines the subject header. Without the carat, any line of the body with the word "Subject" in it would begin the match, and if that same line had [MyLinuxGroup], then it would match, even though it's not the subject line. So a word to the wise, don't forget the carat if you're matching a header line.

Another word to the wise: Procmail regex is case INsensitive. If there's a way to make it case sensitive, I don't know it.

Bracketed group names in the subject are probably the simplest identifiers. The second simplest is an email address. Take this, for instance:
* ^From.*sam.stalker@samstalker.com
I'm sure you can guess the destination: /dev/null.

Sometimes you combine the preceding two in an and condition, like this:
* ^Subject.*\[MyLinuxGroup\]
* ^From.*peter@politicalpeter.com
Every LUG has one: The guy who always talks politics (call him Peter Political). On and on and on, forever and ever, always all the way right or all the way left, and one day you just realize you're wasting too much time reading and responding to him. But here's the thing: He's actually a nice guy and an intelligent guy, especially when communication is one to one. So you blow off your LUG mail from Peter, but not mail from other mailing lists or personal mail from him. In such a situation, you do the anded two line regex that precedes this paragraph.

Speaking of identifying the LUG, the following line identifies every message from MyLinuxGroup so you can place them in the directory for that group:
* ^Subject.*\[MyLinuxGroup\]
ALWAYS, ALWAYS, ALWAYS escape the brackets with backslashes, or it will match many more emails than just those from MyLinuxGroup.

Anyway, I love when the mailing list puts its name in brackets on the subject line. That makes it so easy to detect where it's from and route accordingly. But some lists refuse to do that. Oh, well, there's a regex for the guys who won't put a bracketed name in the subject line, and it looks like this:
* ^(To|Cc).*ourlist@nobracketlinuxgroup.com
Note that the items in the parentheses, separated by a pipe symbol, form an or condition, so what this says in English is that if the email was sent to or CC'ed to ourlist@nobracketlinuxgroup.com, it matches. And of course this can be and combined with another line with a from address to filter a specific person when that person writes to that mailing list, but accept his email otherwise.

So far we've done and with multiple lines and or with parens and a pipe symbol. But sometimes you want to do more complex ors. I think the best and most readable way to do that is to use the large number score technique. I don't even pretend to understand it, but here's the one I have to send all my email from all the different trade magazines I receive to my one magazine folder:
* 9876543210^0 ^From.*pcmag.com
* 9876543210^0 ^From.*itworld.com
* 9876543210^0 ^From.*networkworld.info
* 9876543210^0 ^From.*infoworld.com
* 9876543210^0 ^From.*whatsnewnow.com
* 9876543210^0 ^From.*eweek.com
* 9876543210^0 ^From.*computerworld.com
The preceding vary from simpler Procmail regular expressions in that, before the regular expression proper, they place a huge number, a carat, and then a zero. Because this number is so huge, this short circuit logics -- it quits testing when it finds one of them that matches. If that leftmost number were smaller, it would keep testing even after it found one that matches, which is inefficient. So if you're getting a lot of email into something like this, place the most frequent occurrence at the top, second most frequent second from the top, and the least frequent at the bottom.

I've just scratched the surface of Procmail regex, but it's enough for you to get the right mail in the right folders, and send the undesireable mail to /dev/null.

Destination

The destination of a recipe is, in the simplest case, where you want put the file. It's a line that doesn't begin with a colon or an asterisk, so that's how you know it's a destination, and how Procmail knows it's a destination. Look at each of the following:
/home/slitt/mbox/mylug
/home/slitt/maildir/myotherlug/
Note that the maildir destination has a trailing slash, the mbox one doesn't.

Destinations can also be computer programs. What I'm about to show you is against the law, so don't do it, but I've always had a secret desire to do this to spammers whose real email addresses I've found:
:0:
* ^From.*Bogusproducts@samspammer.com
| send_a_thousand_copies_to_sammys_real_email.sh
When a destination begins with a pipe symbol, that means you pass the email into the stdin of the program. Another handy use for this is to keep a log of the times and subjects of emails that your /dev/nulled people have sent. Another practical use is if you have an old email address that's depricated and soon will be no more, you can pipe the email to a program that tells the sender you received it, but from now on use your new email address. There's no end to what you can do with programs as destinations.

Another type of destination is a simple forward. Let's say you're going with your wife to a deserted island with no Internet, so you want all your email to go to your business partner so he can handle it in real time. You could do this:
:0:
* ^To.*me@mycompany.com
! PeterPartner@mycompany.com
Now you can go on vacation in confidence, knowing that Peter Partner will handle all your email til you get back.

There's one more kind of destination -- a compound recipe destination. Here, the destination is several recipes. Here's one use case. Your Linux group, MyLinuxGroup, votes by email for their favorite presentation topic for next month. The following  complex recipe would first count the vote using a shellscript, and then deposit the email in its proper place:
:0:
* ^(To|Cc).*presentationvote@MyLinuxGroup.org
{
  :0 c
  | record_vote.sh

  :0
  mylinuxgroup.votes/
}
In the preceding, when someone emails presentationvote@MyLinuxGroup.org with an email telling what presentation topic he'd like, it's first submitted to record_vote.sh, and then deposited in Maildir directory mylinuxgroup.votes. Do you see the "c" following the zero in the first subrecipe of the main recipe's destination? That "c" means "keep going even if I match", so that after doing record_vote.sh, instead of doing the normal thing and quitting, it keeps going to the other subrecipe(s).
Steve Litt is the author of the Troubleshooting Techniques of the Successful Technologist. Steve can be reached at his email address.

Procmail Tips and Landmines

What's wrong with this recipe?
:0:
* ^Subject.*[MyLinuxGroup]
.mylinuxgroup/
If you have the preceding recipe, you in a heapa trouble son! The preceding matches any email whose subject contains i, l, m, n, o, p, u or y, or several other letters, because the brackets in the Regex denote a set of letters, not a string. If you want to match an actual string bracketed by brackets, you need to escape those brackets with backslashes, like this:
:0:
* ^Subject.*\[MyLinuxGroup\]
.mylinuxgroup/
Without the backslashes, you'll find your .mylinuxgroup folder filling up with all sorts of junk. What would be worse is if you were redirecting subjects containing [eat your vitamins] to /dev/null. Without the backslashes, every email with subject containing a, e, i, o, u, y, several consonants, or space would be irretrievably gone if it hadn't matched an earlier recipe. Can you think of a subject line that wouldn't contain a vowel or a space? Neither can I. And once something's sent to /dev/null, you don't know, you have no record, it's forever, irretrievably gone. Backslash those brackets!

This brings up another question. Are you absolutely sure you want to send anything straight to /dev/null? No doubt it's Unixly hip. No doubt it's the highest performance. No doubt if you were a REAL Unix administrator you'd never make a mistake that would cause redirecting to /dev/null a problem. Right now I'm redirecting to /dev/null because my top priority is getting my newly transitioned email system working well and under control. But some day I'm going to build a facility that logs everything before it gets sent out to /dev/null. That way I can look at the log and see if anything looks odd. Better yet, if I suspect something's missing, I can look at the log and see if that something got /dev/nulled by Procmail, and if so re-contact the sender. I can think of two ways to make such a log. The first is this:
:0:
* ^Subject.*Grow it bigger, she will love it
| write_one_devnull_record.sh
Pretty slick, huh? One problem. As far as I can see, the preceding stops (blocks) until write_one_devnull_record.sh completes its work. That can seriously slow down email retrieval, especially after you return from a trip and emails have piled up.

Another possibility is to write the email to a FIFO for processing by a daemon:
:0:
* ^Subject.*Grow it bigger, she will love it
/home/slitt/devnull.fifo
I think there's an art to what you have at the bottom of .procmailrc. For instance, my last recipe is this:
:0:
* ^
$DEFAULT
The preceding puts everything into the $DEFAULT directory. I know, I know, this should happen anyway, but just for good measure, I leave this at the bottom. And here's my second to last:
:0:
* ^Subject.*\[aeiouy\]
.litttest.aeiouy/
If I send  a subject [aeiouy] to myself and it *doesn't* end up in .litttest.aeiouy, then something else has illegitimately grabbed it before and I need to investigate. Or else something else isn't working.

Procmail is not without its problems. One wrong recipe can reroute most or all of your email. You need to be careful and eternally vigilant. There are other LDA's besides Procmail. There's even one that comes with Dovecot. Me -- for the time being I'm sticking with Procmail -- it's ubiquitous, and extremely easy to clone existing filters with a text editor.

I've only scratched the surface, but with this article you have enough info to build a pretty functional Procmail system.
Steve Litt is the author of the Troubleshooting Techniques of the Successful Technologist. Steve can be reached at his email address.

My New Architecture

By Steve Litt
Remember this diagram?
Generic Email Architecture
Typical email between individuals


I used fetchmail to POP3 pull email off my ISP, then Fetchmail handed off to Procmail to do the local delivery to mbox file /var/spool/mail/slitt. My Kmail was set up to grab mail from /var/spool/mail/slitt, filter it internally, and place it in Kmail's mailbox hierarchy. Here's a diagram of the preceding that's more focused on my desktop:

Desktop Email Architecture: Kmail Handles Filters and Folders
Old setup with kmail

In the preceding, all filtering and sorting was done within Kmail, the email client, and the message folder tree was kept within Kmail. One stop shopping -- great as long as Kmail worked for me. The downside was that this architecture depended completely on Kmail, and when Kmail completely entwined itself with Akonadi, thereby making it unusable for my purposes, I had a problem. So I converted to the following Dovecot-centric architecture:
Desktop Email Architecture: Procmail Handles Filters, Dovecot Handles Folders
Diagram of the Dovecot-centric setup

Notice in the preceding, the Dovecot IMAP server handles all message folders, instead of letting the email client handle them. Notice that in this architecture, Procmail with its filters handles all filtering, instead of letting the email client do the filtering. This leaves the email client as nothing more than a viewport to the Dovecot IMAP server. At that point I chose my email client exclusively for its abilities as a viewport. No need to consider mailbox formats or filtering options -- those are all done at the LDA and IMAP level. I chose Claws-Mail as what I feel is the best viewport to IMAP.
Steve Litt is the author of the The Key to Everyday Excellence. Steve can be reached at his email address.

Claws-Mail

By Steve Litt
Once my filtering and storage were done in Dovecot, my choice of email clients expanded to anything that could completely and reliably interface with IMAP. That's a pretty big list. Choosing was a non-trivial task.

My old friend Sylpheed took minutes or hours to change from threaded view to unthreaded view in a large directory. One less to consider. Kmail worked just fine in IMAP mode, but I hadn't come all this way to stick with Kmail (and Akonadi). The GNUMail program seemed quirky, crashy, and had an inconvenient user interface. Another one bytes the dust.  Balsa seemed to have some problems reliably reflecting what was in IMAP. Actually, they all did, but I eliminated Balsa. That left Thunderbird and Claws-Mail still standing. Both had minor IMAP problems, both were well suited to be my new email client.

Moving and deleting folders can be a challenge in Thunderbird, but as long as you realize the risk and check the actual Dovecot structure when moving or deleting, everything should be fine. I haven't yet experienced a similar problem in Claws-Mail.

In the end I chose Claws-Mail because of a suspicion that something from the folks who made Firefox just might be bloated, and I had just walked through the fires of perdition to get away from bloat. Claws-Mail turned out to be an excellent choice. I've used it for several weeks now, and like it more every day. Claws-Mail sometimes fails to recognize folders added by other email clients, which of course is a big issue while you're converting away from Kmail. No problem, you can just collapse the entire IMAP tree, right click, choose "check for new folders", and the new folders will be visible.

Claws-Mail has an active and helpful mailing list with polite and knowledgeable people. All in all, Claws-Mail plus Dovecot has turned out to be an excellent choice.

Claws-Mail Tips

Here are a few Claws-Mail tips...

Two Deletion Modes

One deletion mode is like most email clients. You click the message, click move to trash or keyboard it, and the message gets moved to the trash folder you've specified in your setup. If you want to put it back, you right click it in the trash folder, choose "move", and navigate to whatever folder it came from or whatever folder you want it in.

In the other deletion mode, when you say "move to trash", the message becomes light gray and becomes marked, but stays right where it was in the original folder. If you want to undelete it, you highlight it and say "unmark". Easiest way to unmark is the U keystroke. Because the Claws-Mail message folder doesn't have Edit->Undo to undo an unwise deletion, I like the grayed out mode better, as a lot of times I delete something and then instantly realize it wasn't something that should be deleted. Meanwhile, if you find grayed out messages are distracting you, you can can View->Hide_deleted_messages or its keystroke equivalent, Ctrl+Shift+Z (that's the default keyboard shortcut for this action). Then, if you want to again see the deleted ones, you can Ctrl+Shift+Z again. After a few days, the grayed out messages disappear forever, so they don't build up.

The way you switch between these modes is you check the "Move deleted mails to trash and expunge immediately" checkbox on the account preferences "receive" tab to enable the mode where move to trash really means move to trash, and uncheck it to enable the mode where move to trash means mark and light gray it.

Folder Coordination

Every few days, collapse your local IMAP account, right click it, and say "rebuild folder tree." This rebuilds the folder tree, getting what you see to be what the IMAP server is serving out, and it also deletes cache on all folders. Deleting Claws-Mail cache is usually a good thing when you're using IMAP and you don't use it offline. You don't want to use double disk space -- one for IMAP messages and one for Claws-Mail cache.

Whenever you create or change folders without using Claws-Mail, collapse your local IMAP account, right click it, and say "check for new folders." The newly created folders appear in the right place.

Creating New Keyboard Shortcuts

You can create a hotkey combo for almost anything on the menu. To do so, hover over the menu item so it's highlighted, and then just press the desired keystroke combo (make sure it's not used somewhere else). If there's a way to add new menu items, I don't know it yet.

Steve Litt is the author of the Troubleshooting: Just the Facts. Steve can be reached at his email address.

SSL

Did you notice a big, gaping hole in everything discussed so far in this document? So did I. It's called password security. As the preceding documentation stands, if I'm on my LAN and access my Dovecot IMAP server (which is on 192.168.100.2) from a different machine, my password goes over the wire in clear text. If I were ever to pinhole my firewall to access that IMAP server from out on the Internet, my password would travel clear text all the way. And that's the password to my main desktop computer. Not an optimal situation.

No problem. Dovecot supports secure IMAP with SSL, so I changed it to work with IMAP SSL. Here's how -- it involved only a few easy adjustments to /etc/dovecot/dovecot.conf and to the client (in this case Claws-Mail)...

My first step was, on 192.168.100.2, to do an nmap command:
slitt@mydesk:~$ nmap 192.168.100.2

Starting Nmap 5.21 ( http://nmap.org ) at 2012-02-16 11:39 EST
Nmap scan report for www.domain.cxm (192.168.100.2)
Host is up (0.00028s latency).
Not shown: 995 closed ports
PORT    STATE SERVICE
22/tcp  open  ssh
53/tcp  open  domain
80/tcp  open  http
143/tcp open  imap
993/tcp open  imaps


Nmap done: 1 IP address (1 host up) scanned in 0.15 seconds
slitt@mydesk:~$
Referring to the preceding, the good news is that secure IMAP appears to already be working, or at least listening, on port 993. The bad news is that port 143 is available for non-SSL logins. So, within /etc/dovecot/dovecot.conf, I did the following:
I then restarted Dovecot, and nmap showed that port 993 was still there, but port 143 was gone. Good deal!

As you can imagine, the preceding broke the email clients I had previously connected to my IMAP server, but it was easy enough to fix. Here are my adjustments to my Claws-Mail clients' account preferences for the account for the IMAP on 192.168.100.2. I adjusted one client on 192.168.100.2 and one on 192.168.100.11:
With everything set up this way, if I want to, I can pinhole my firewall and hit my home IMAP from the road, using SSL for the protection of my password. Pretty cool, huh?
Steve Litt is the author of the Universal Troubleshooting Process Courseware. Steve can be reached at his email address.

Steve, You Sound Angry

By Steve Litt
I guess I am. But can you blame me when the developers "improve" a perfectly functional program to the point of unusability? What were they thinking? Every year they tie together more junk into their software monolith, and every year it gets shakier. Remember V'Ger from Star Trek: The Movie? Kinda like that -- a bumbling buncha space junk that for the most part works, but then kills you. If you go along with it this year, what else will KDE accumulate next year?

And then there's this:
slitt@mydesk:~$ ls -lh .kde/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.db 
-rw------- 1 slitt slitt 1.6G 2011-12-12 16:20 .kde/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.db
slitt@mydesk:~$
Nuf said? And just to make life more exciting, there are the dbus-daemon and nepomuk instances that go rogue and take 99% of the CPU. After using the svelte simplicity of Claws-Mail for a couple days, you bet I was mad at the Kmail developers for what they put me through for eleven years, after which they duct-taped Kmail2 to Akonadi, offered a crashing one-and-only-one-time conversion program, and basically laughed at my plight.

But that's all over now. Kmail's out of my life, and I anticipate completely removing KDE from my machine by the end of the month. No more Nepomuk.
Steve Litt is the author of the Samba Unleashed. Steve can be reached at his email address.

Escape From Kmail

Kmail had me painted into a corner. Tens of thousands of emails I needed to keep, some in maildir, some in mbox files of various flavors, no visible method of conversion to other clients, and then for Kmail2 on Ubuntu 11.10 the developers invalidated their storage mechanism and gave only a nonfunctional one-try-only conversion.

Want to know a secret? Transitioning out of Kmail promised to be so time-consuming and difficult that I considered finding a way to re-convert, and use Kmail2 with its Akonadi database. But I'd been here before, and knew better.

"Before" was around the turn of the century, and instead of Kmail the software was Windows. But it was a similar situation: Get out now, or forever accept whatever horrors the vendor later sends your way. A horrible law called UCITA was on the horizon, threatening to make migration away from Windows impossible. Windows was about to get Internet license validation, which could prevent installation on a new machine if the old machine broke, or could even prevent re-installation on the same machine if your Windows got messed up to the point of needing reinstallation. I got out not while the getting was good, but while the getting was at all possible.

It's called vendor lock-in. It happens when migration away from your software becomes too costly to consider. Most vendor lock-in is done on purpose, for a profit motive. That's the explanation for Windows. With KDE, who knows? My guess would be ego -- building the biggest monolithic edifice in the history of free software. But in the end motive doesn't matter -- if you stick any longer you won't be able to escape at all, so you bail.

It took me over a week to learn everything and do everything. Escaping from Kmail was almost as big a job as escaping from Windows. It was costly indeed. But I asked myself "If I continue with Kmail, who owns my data?" From every angle I could see, once I switched to Akonadi-encumbered Kmail, the answer would have been "Kmail does." That's not how I live my life, so I took the time to convert to Dovecot/Claws-Mail. Now when asked who owns my data, I respond "I do!", and that's just the way I like it.

The conversion was time consuming and costly for me because beforehand I had no idea what to convert to or how to run the conversion. Thanks to this document, you're in a much better position. This document enables you to escape from Kmail quite easily.

Life After Windows: The End of an Era

By Steve Litt
So here we are in 2012, with a regular column called "Life After Windows." The funny thing is, at this point I hardly remember what it was like to use Windows. My last regular Windows use was Windows 98 in March 2001. I can't visicerally remember what it was like using windows. Or what it was like before Linux. So it's kind of silly keeping a column called "Life After Windows", now that it's been eleven years. So Linux Productivity Magazine goes on, but this is the last "Life After Windows" column.
Life After Windows was a regular Linux Productivity Magazine column, by Steve Litt, bringing you observations and tips subsequent to Troubleshooters.Com's Windows to Linux conversion.
Steve Litt is the founder and acting president of Greater Orlando Linux User Group (GoLUG).   Steve can be reached at his email address.

GNU/Linux, open source and free software

By Steve Litt
Linux is a kernel. The operating system often described as "Linux" is that kernel combined with software from many different sources. One of the most prominent, and oldest of those sources, is the GNU project.

"GNU/Linux" is probably the most accurate moniker one can give to this operating system. Please be aware that in all of Troubleshooters.Com, when I say "Linux" I really mean "GNU/Linux". I completely believe that without the GNU project, without the GNU Manifesto and the GNU/GPL license it spawned, the operating system the press calls "Linux" never would have happened.

I'm part of the press and there are times when it's easier to say "Linux" than explain to certain audiences that "GNU/Linux" is the same as what the press calls "Linux". So I abbreviate. Additionally, I abbreviate in the same way one might abbreviate the name of a multi-partner law firm. But make no mistake about it. In any article in Troubleshooting Professional Magazine, in the whole of Troubleshooters.Com, and even in the technical books I write, when I say "Linux", I mean "GNU/Linux".

There are those who think FSF is making too big a deal of this. Nothing could be farther from the truth. The GNU General Public License, combined with Richard Stallman's GNU Manifesto and the resulting GNU-GPL License, are the only reason we can enjoy this wonderful alternative to proprietary operating systems, and the only reason proprietary operating systems aren't even more flaky than they are now. 

For practical purposes, the license requirements of "free software" and "open source" are almost identical. Generally speaking, a license that complies with one complies with the other. The difference between these two is a difference in philosophy. The "free software" crowd believes the most important aspect is freedom. The "open source" crowd believes the most important aspect is the practical marketplace advantage that freedom produces.

I think they're both right. I wouldn't use the software without the freedom guaranteeing me the right to improve the software, and the guarantee that my improvements will not later be withheld from me. Freedom is essential. And so are the practical benefits. Because tens of thousands of programmers feel the way I do, huge amounts of free software/open source is available, and its quality exceeds that of most proprietary software.

In summary, I use the terms "Linux" and "GNU/Linux" interchangably, with the former being an abbreviation for the latter. I usually use the terms "free software" and "open source" interchangably, as from a licensing perspective they're very similar. Occasionally I'll prefer one or the other depending if I'm writing about freedom, or business advantage.
Steve Litt has used GNU/Linux since 1998, and written about it since 1999. Steve can be reached at his email address.

Letters to the Editor

All letters become the property of the publisher (Steve Litt), and may be edited for clarity or brevity. We especially welcome additions, clarifications, corrections or flames from vendors whose products have been reviewed in this magazine. We reserve the right to not publish letters we deem in bad taste (bad language, obscenity, hate, lewd, violence, etc.).
Submit letters to the editor to Steve Litt's email address, and be sure the subject reads "Letter to the Editor". We regret that we cannot return your letter, so please make a copy of it for future reference.

How to Submit an Article

We anticipate two to five articles per issue. We look for articles that pertain to the GNU/Linux or open source. This can be done as an essay, with humor, with a case study, or some other literary device. A Troubleshooting poem would be nice. Submissions may mention a specific product, but must be useful without the purchase of that product. Content must greatly overpower advertising. Submissions should be between 250 and 2000 words long.

Any article submitted to Linux Productivity Magazine must be licensed with the Open Publication License, which you can view at http://opencontent.org/openpub/. At your option you may elect the option to prohibit substantive modifications. However, in order to publish your article in Linux Productivity Magazine, you must decline the option to prohibit commercial use, because Linux Productivity Magazine is a commercial publication.

Obviously, you must be the copyright holder and must be legally able to so license the article. We do not currently pay for articles.

Troubleshooters.Com reserves the right to edit any submission for clarity or brevity, within the scope of the Open Publication License. If you elect to prohibit substantive modifications, we may elect to place editors notes outside of your material, or reject the submission, or send it back for modification. Any published article will include a two sentence description of the author, a hypertext link to his or her email, and a phone number if desired. Upon request, we will include a hypertext link, at the end of the magazine issue, to the author's website, providing that website meets the Troubleshooters.Com criteria for links and that the author's website first links to Troubleshooters.Com. Authors: please understand we can't place hyperlinks inside articles. If we did, only the first article would be read, and we can't place every article first.

Submissions should be emailed to Steve Litt's email address, with subject line Article Submission. The first paragraph of your message should read as follows (unless other arrangements are previously made in writing):

Copyright (c) 2003 by <your name>. This material may be distributed only subject to the terms and conditions set forth in the Open Publication License, version  Draft v1.0, 8 June 1999 (Available at http://www.troubleshooters.com/openpub04.txt/ (wordwrapped for readability at http://www.troubleshooters.com/openpub04_wrapped.txt). The latest version is presently available at  http://www.opencontent.org/openpub/).

Open Publication License Option A [ is | is not] elected, so this document [may | may not] be modified. Option B is not elected, so this material may be published for commercial purposes.

After that paragraph, write the title, text of the article, and a two sentence description of the author.

Why not Draft v1.0, 8 June 1999 OR LATER

The Open Publication License recommends using the word "or later" to describe the version of the license. That is unacceptable for Troubleshooting Professional Magazine because we do not know the provisions of that newer version, so it makes no sense to commit to it. We all hope later versions will be better, but there's always a chance that leadership will change. We cannot take the chance that the disclaimer of warranty will be dropped in a later version. 

Trademarks

All trademarks are the property of their respective owners. Troubleshooters.Com(R) is a registered trademark of Steve Litt.

URLs Mentioned in this Issue


_