TextMate News

Anything vaguely related to TextMate and macOS.

My Wishes for an Email Application

Martin Winter asked me what I meant by “someone should write a decent mail application” so let me clarify. The Hawk Wings blog already has two dozen interviews with prominent Mac people about Mail, so this is just my point of view, not claiming to be original or anything.

I also know that I could get some of this if I used other email programs. But frankly, Mail is still the best option for me.

Threading

I think most people who deal with high traffic mailing lists would like this. Mail does grouping based on subject (likely by design), but this is not the real thing.

Grouping makes too many mistakes, both with unrelated letters that share the same subject or on mailing lists where a subject is sometimes changed e.g. from: Re: [TxMt] something to [TxMt] Re: something.

Real threading (based on in-reply-to headers) would show the thread as the true tree it is, so that you can see from the message overview alone, to whom new letters are replying.

It would also allow for features such as “Kill Offspring” or “Kill Siblings” which can be used to quickly prune a thread into only the interesting branches.

Tagging

As countless of web applications have shown, tagging is useful. I could easily do without a dedicated ticket system if I could tag incoming mail with BUG, CRASH, REQUEST, FIXED (and Mail handled the in-reply-to stuff).

Of course there are other uses for tagging, e.g. tagging interesting letters on mailing lists. Currently Mail supports flagging them, but some stuff is “I should blog this”-interesting, and other is “I should link to this when people ask support questions”-interesting.

Searching

I often have to search my mail boxes, and even though Spotlight is fast, it generally takes too long. Why? Because I get far to many results. What I want is the ability to provide multiple match conditions. A simple date interval alone would be a tremendous help.

I can do this with smart mail boxes, but even these are lacking in features, e.g. I can’t match arbitrary message headers, and as others have ranted on before me, it is either “match all these conditions” or “match any of these conditions”. There is no way to do boolean logic.

Mail Templates

When doing support, I write a fair amount of identical text. Mail has no way to help me with that.

Needless to say that something like snippets would be wonderful in Mail. It should also do a better job at controlling how the initial reply looks. I have read rants about this before, but it is really weird how Mail sets up things, with an initial empty line that holds the insertion point, and then the signature below the quoted text. I mean, if it was setup for top-posting, shouldn’t the signature be above the letter?

Often I end up re-typing my signature, because the last 10+ lines of the quoted text needs to be deleted, and here it is simply easier to use ⇧↘ + ⌫ (that’s select to end of letter and delete) and re-type it.

I realize here though that what I really want is just better support for external editors, and with the Edit in TextMate input manager, I have gotten that. In TextMate I have all the snippets I need, and I am even toying with a command that reformats the pristine reply, as produced by Mail, into how I would like it to look.

Email Address Mapping

It seems to me that some behavior can be controlled based on email address patterns. For example, when I reply to someone who uses a *.dk email address, I most certainly do not want the English spell checker enabled. I would also like the “On XYZ someone wrote:” replaced with a more informal greeting.

I also want to setup how email addresses should be mapped when replying to a letter. For example when people write to one of my outdated email addresses, I want the reply to be from my current. There are also a few other situations where I do not want to reply from the email address to which the letter is sent.

Speaking of mapping, many mailing lists are setup so that only subscribers can post, and many subscribers have multiple email addresses. Being able to indicate that messages sent to a particular mailing list should use a given from address, would be useful.

In addition, when I hit “New Message” the To: field should be pre-filled with the address corresponding to the currently selected message box (mailing list). This would need to be a per message box property.

Message Display

I have seen many attempts of smarter message display. For example Google knows when to hide quoted text (and make it into a link). How the message should be displayed is likely a very personal thing, and subject to tweaking a lot of heuristics.

So in my ideal email application, the message would be displayed with WebKit and piped through a user controllable chain of filters (plug-ins/scripts).

Then I could do a simple sed filter to highlight various keywords, or I could pipe letters through Markdown :)

I am thinking that formatting the reply to a letter, could use a similar system.

Extensibility

The key point from the previous section is that the user should be able to shape his email program.

I think we are many who deal with high volumes of email, and being able to better shape our email client would save us a lot of work.

For example searching through the offline archives of the TextMate mailing list is faster and generally gives better results than searching online, but that means I need to transform the offline match to an URL which points to the archive (something I do a lot).

It would be nice if this transformation was automatically done when displaying messages from the mailing list. Something that the scheme outlined above would allow. Think GreaseMonkey for your mailer.

Naturally sharing extensions should be dead simple, and so should the creation of these be. I have mentioned a few different things that I want to customize, but an attempt should be made to unify this into one system. A bit like how TextMate has 3 parts to customization: 1) when should this be active (the scope selector), 2) how should it be activated (the actual activation mechanism, be it a hotkey or mouse gesture), and 3) the actual action (running a command, inserting a snippet, or just changing a preference value).

Conclusion

I know Apple’s Mail does most things for most people, is free, and comes bundled with the operating system. Heck, I mostly enjoy using it myself, despite its delightful quirks such as showing duplicates of each letter (IMAP support), the compose window which can’t handle tabs (making majority of the patches I receive useless), the weird “end of document” behavior, swallowing half my letter when I switch the from email address (because it thinks it was my signature, which is tied to account), etc.

But I do think there is a lot of power users out there, like me, who long for an email application which can be a little smarter in handling our email related tasks.

If you have anything to add, please do so in the comments, as I have almost successfully convinced a friend to work on this.

categories General

46 Comments

07 May 2006

by Matthew Johnson

You can do boolean logic using multiple smart mailboxes via the In Mailbox and Not In Mailbox criteria. It’s a hack and is much more complicated than it should be, but it will work if you don’t need to do boolean logic that’s too complex. I don’t use this in Mail, but it’s identical to how iTunes does smart playlists and I use it quite a bit there…

This describes much of what I’ve been looking for, and would be something I’d pay quite a bit for if it were done well.

A couple of minor other things I’ve been missing from the last several mail apps I’ve used:

  • “Pre-file tagging”: automatically applying mailbox tags on the way in using a rules system, and having a single-stroke file-to-tagged-mailbox when reading (for mail that belongs in “primary” flow, but can usually be quickly dispensed with).

  • Keyboard entry for filing: again, a way to file messages without the dragging required in Apple Mail. Presumably some keystroke that would pop up a field that does Quicksilver/LaunchBar-style mailbox selection.

I’ve tried other clients, but they tend to be lacking in other ways; most commonly, no decent offline IMAP support; and Thunderbird still feels cumbersome.

So, yes, encouragement is provided.

I have read rants about this before, but it is really weird how Mail sets up things, with an initial empty line that holds the insertion point, and then the signature below the quoted text. I mean, if it was setup for top-posting, shouldn’t the signature be above the letter?

Unless I am misunderstanding you - the placing of signatures is a preference:
Mail -> Preferences -> Signatures -> “Place Signature above Quoted Text”

Some good points, but a lot of them sound like they should be features in a dedicated “mailing-list client” sort of app, rather than in a general mail application.

@Lee

Keyboard entry for filing: again, a way to file messages without the dragging required in Apple Mail. Presumably some keystroke that would pop up a field that does Quicksilver/LaunchBar-style mailbox selection.

Have you looked at MsilActOn and MailTags? (http://www.indev.ca/)
I have found them invaluable.

Needless to say, native support would be much better.

@Allan

If your friend is doing this as a side project, and would like some collaborators, I would like to volunteer myself, depending on the programming language used.

And in any case I would so totally use such an app, if it was done properly.

I couldn’t agree more.
I’d be willing to wager a pretty hefty sum that there’s a huge market out there of power users who’d like something more powerful than Mail but but gag on the thought of using, say, Mailsmith.

Sounds like GyazMail

http://www.gyazsquare.com/gyazmail/

“Sounds like GyazMail”

No, it does not.

GyazMail Future Plans:
IMAP support
Spotlight support

every thought of using a trouble ticket system? Then you dont need Templates inside a mail program. State of the art support systems have plenty of features in that area.

Ooops. I guess IMAP is the deal breaker.

But there is:
Ability to search messages across multiple folders and with multiple criteria

  • support for regular expressions.

I continue to hope that apps will take advantage of tagging and searching technologies by intelligently suggesting tags based on context. A photo management application would have object and facial recognition technology so that it knows when it’s a picture of a sunset, outdoors, or your friend Bob. When the pictures are imported it would present you with the proposed tags.

The ultimate email app would handle this by looking at the subject, message text, sender, recipients, and previous messages in the thread. When you read a new message it would present you with suggested tags, and you could accept or edit them as you wish. The algorithm would learn from your selections in a similar manner as junk mail filters, so that future tag / keyword suggestions are more accurate.

Why is this so important? Like with most technologies, training the user is the most important step of realizing its power. By suggesting tags and making this process more automated, tagging messages takes less time for the power user, convinces the moderate user to go ahead and use the technology, and simplifies the interface and associated workflows.

Good luck with this project, I’ll be looking forward to reading more about it.

Matthew Johnson: Yes, though not really the same as quick on-the-fly searches using boolean logic :)

Andrew: I hadn’t seen that option. That does make more sense. I wonder why it is not on by default, if top posting is their intent. Of course it doesn’t help me.

As for a “dedicated mailing list client” that is probably what a usenet client is. However, I don’t see the advantage of splitting this into two applications, as regular email and mailing list email overlap in basically all other aspects than the threading, but even that is not entirely so, as I sometimes take part in group discussions/conversations where half a dozen people are in via cc’ed.

Labrat: Yes, lack of IMAP is a serious deal breaker.

Marc Logemann: Using a dedicated ticket system has too many downsides (for me), and I also need mail templates/snippets for other than direct support email. For example people write postmaster@… to ask why their post got denied (though which I have now setup auto-reply for), I sometimes have to give the same text/URL/etc. on mailing lists, in private letters etc.

Though I can also see the reasoning of doing this more globally, e.g. like Textpander, as the same snippets are likely useful in other contexts.

For Mail.app templates I was messing around with this for a while:

http://www.mactank.com/mailtemplate/howto.html

07 May 2006

by joachimm

Personally I have high hopes for the Kiwi-mail client from Matt Ronge[http://theronge.com/]. It is probably still far away time wise. Since Matt is a TextMate user I know that he has great taste :) and that it will be worth waiting for.

07 May 2006

by Håkan W

Sounds like what you want is basically Thunderbird. FYI, a Mail.app importer is in the works.

/Håkan W

07 May 2006

by Phill Kenoyer

Mutt has everything that you’d ever want, but it does not have a GUI. It would be awesome if someone would build a Cocoa version of Mutt. It would instantly be the best email program out there.

I miss the one feature of Mutt that I can set my reply-to address based on the from address.

07 May 2006

by Chris

There’s two big unsurmountable reasons for me that I can’t use Mail.

The first one, and I don’t understand why it doesn’t bother other people?, is that Mail automatically gets (or tries to get) your mail on launch. I really hate apps that decide how I should work, without offering any preference to change.

The second is no easy ability to use the same email addresses from different SMTP servers. I use a PowerBook, and connect from many locations, and have to use any one of three main SMTP servers depending where I’m connected.

This would be a brilliant application of a newly open-source TextMate editor core framework, when that (hopefully) comes to pass. I have to say, working with TextMate has finally brought me to the point of understanding why Emacs weenies (and I almost thought I was one at one point) are so enthusiastic about Gnus and the like: when you finally find the perfect editor, it’s almost painful to be stuck using something else. I appreciate that this editor-centric philosophy (coupled with massive existing infrastructure) was a big advantage for BareBones in making MailSmith – I just wish the same could be applied to a set of tools I love in the way others love BBEdit, much as I completely fail to relate to BareBones products, myself.

Where can I place my order? ;-)

Keep fingers crossed for Leopard I suppose–they have made steady improvements in the past. You never know what you’re missing until you get it like Textmate shows.

“Often I end up re-typing my signature, because the last 10+ lines of the quoted text needs to be deleted, and here it is simply easier to use ⇧↘ + ⌫ (that’s select to end of letter and delete) and re-type it.”

If you have more than 1 signature a pop up menu appears under the subject line. Pulling up this menu, and selecting the same signature again will re-type it for you. If you don’t have this menu, you might need to add another signature.

BTW, is it just me or is the signatures config panel really confusing?!

“When doing support, I write a fair amount of identical text. Mail has no way to help me with that.”

Could you set up multiple signatures with your various templates in them?

The one feature I would really love to see is a vertical screen split, dare I say it, like Outlook. Mailboxes on the left, messages in the middle, and current message on the right. Perfect for all these wide screens.

Keep up the good work.

Yeah, that would be gnus. I kinda miss it, but I also kinda like the prettiness of Mail.app and it sorta does all I need. Now if somebody was to come up with a lovely cocoa front-end for Gnus (not just an aqua version of XEmacs, but a full cocoa GUI for gnus itself), well, that would just be the ultimate mail/news reader!

Kiwi is looking pretty impressive!

I have an idea… how about Matt Ronge and Allan team up and create the ultimate OS X mail client?

Hey Allan, it’s UnLogikal from the IRC channel. Just wanted to mention that I am indeed working on such a mail client as I discussed with you. It now has my “almost” full attention and while going through some rewrites that I seem to have come up with a better way of doing things, is still very very not ready for use … but. I am at least trying to tackle this problem.

Anyone that wants more information, I have a website, clickie my name above and you’ll goto my extremely basic text page related to it, including a link to my forum where I’d be happy to discuss more stuff.

Sorry, no screenshots, the interface is a tad different than your traditional email client and I think it works better, I’d prefer to keep it to myself until I’m much closer to release.. hopefully this year.

Kyle

A lot of what you’re looking for can be added through third-party utilities. Examples:

Tagging: Mail Tags

Searching: Matt Neubergs’s NotLight can handle complex searches

Threading: Mail actually only uses the subject line as a fallback. It uses in-reply to or some other header (message iDs on mailing lists? I dunno) to handle threads. I know this because I end up with a lot of messages in the same thread on mailing lists after someone replies to a list message, deletes the whole thing, and tries to start a new thread.

Templates: Look at a tool like CopyPaste or TypeIt4Me to handle templates, glossary/auto-text, and other snippet keeping. Since I use the same text in a lot of different apps, I find this far more useful than miring it in a single application.

Mind you, none of this is in defense of mail, I just hope it might ease the pain for some Mail users.

Nik: Thanks, I do however know these things.

Mail Tags is not useful for IMAP. I use Textpander (not TypeIt4Me) as mentioned above, but mostly though I just drop into TextMate, which is only ⌃⌘E away. As for threading, it seems Mail is not consistent from user to user (we’ve had a few discussions about this on IRC, where different people saw threads on the list grouped differently). However, it is still not threading, only grouping, and it has none of the threading features I seek (and the grouping is irritating half the time, when it groups unrelated stuff).

Tim: I have had really bad experiences with signatures, re my “deleting half the letter” (when switching account), so I avoid switching them during a compose. As mentioned above, TextMate is only a key away, so I just drop into that, which is faster than having to grab my mouse, select a signature, and move it up so it is below the proper paragraph etc.

But thanks for the suggestion.

My post was meant to clarify what I meant by “someone should write a decent mail application”. I do know that there are hacks and workarounds for some problems in Mail, and I also know that there are other email clients out there that does half of what I want (but generally then lack half of what I actually like about Mail ;) ).

Mail Tags is not useful for IMAP

Hopefully it will be in the not-too-distant future:

MailTags just keeps getting better and better

Mutt has everything that you’d ever want, but it does not have a GUI. It would be awesome if someone would build a Cocoa version of Mutt. It would instantly be the best email program out there.

Last I checked Mutt’s IMAP wasn’t as well supported as in other clients.

Some consider Mulberry the “gold standard” of IMAP clients. I used it for +5 years before switching to Mail a couple months after Cyrusoft filed for bankruptcy last October(?) and never lost even a single message that I’m aware of. Mail, on the other hand, has definitely been responsible for mysteriously deleting messages. I leave the activity window visible and only move messages between mailboxes if Mail is idle.

Mail’s threading is weak. The Message framework it uses doesn’t support IMAP server threading capabilities.

A closer to ideal OS X mail client for me would be a Mail/Mulberry/Mutt mutant of sorts, e.g. – Mail for its OS X Cocoa goodness; Mulberry for its reliability and some unique, powerful capabilities (e.g. separation of incoming/outgoing accounts, identities); Mutt for its flexible customization.

I’m working on a power user e-mail client known as Kiwi. I’m building components that the application will be built out of, I’ve been working on an IMAP/SMTP framework known as MailCore and a set of classes to make implementing worker threads much easier. First and foremost, I want Kiwi to be reliable, customizable (using Lua!) and fast. There will be a number of features inspired by TextMate, especially with regards to keyboard shortcuts. We’ll see what happens, I can’t promise anything :) Oh, and everything is open source under the BSD license.

“When doing support, I write a fair amount of identical text. Mail has no way to help me with that.”

I use TypeIt4Me. Have you tried it?

The most frustrating things about Mail for me are:

  1. When I send a message I often end up with a copy in Sent and a copy in Drafts (that sometimes I have to delete 2 or 3 times before it actually disappears)! Does anyone else get this?

  2. The way it deletes your message when you change signatures.

  3. Search often doesn’t actually work. I search for something that I know is there and it doesn’t show up. Why??

SatCure: Read my previous comment!

Peter: I have 1+2, and I also have a problem with Mail showing each letter twice. This also seems to be a common problem.

Mutt is great. I use a seperate imap syncing program called offlineimap to sync imap account locally. I use mutt for what it was designed for - reading mail. I use textmate as my editor for composing mutt email messages. I have a wrapper script around offline imap which shows growl notifications when I get a new email message. The wrapper is called from cron every 5 minutes, this means I do not many to click ‘get mail’ anymore as its all automated (although I do have a key binding in mutt to invoke the script). There is pretty nothing that cannot be neavily customized by mutt.

I reccomened using mutt-ng instead of original mutt. mutt-ng includes many patches and feels more modern (thanks to sidebar).

Speaking of mapping, many mailing lists are setup so that only subscribers can post, and many subscribers have multiple email addresses. Being able to indicate that messages sent to a particular mailing list should use a given from address, would be useful.

In addition, when I hit “New Message” the To: field should be pre-filled with the address corresponding to the currently selected message box (mailing list). This would need to be a per message box property.

Mail should do this already… At least, this is the behavior I see.

Ryan: Mail never fills in the To: address. It does have some smartness with respect to sender address, but it is a heuristic. It works fairly well when using New Message for other than mailing lists, as it will use a sender address that the currently selected message box has mail delivered to (i.e. looking at the To: field of mail in that box.) But it breaks for mailing list letters, since the receiver here is generally set to the list. So Mail will not be able to find a match. The same is the case when using Reply (where it first looks at the To: of the letter you reply to, and then possibly falls back on the currently selected message box heuristic.) I don’t have any real insight into how exactly it works, and I didn’t do any thorough tests, but it has never done the correct thing with mailing lists for me. I have however solved the problem by creating a new IMAP account which is only used for mailing lists. This IMAP account only has one sender address associated with it (the one I use for the mailing list), which will force Mail to use that (both for new messages and replies) as it will stay within the current IMAP account when searching for a sender address to use.

You can do some basic booleans using spotlight and also when searching mail. Look around the web for “spotlight boolean”.

You can do stuff like:

Dave(-Chiu +textmate)

http://www.hiram.nl/ipsedixit/artikel/793/boolean-search-in-spotlight

You can also use “kind” keywords:

http://www.apple.com/macosx/tips/spotlight.html

hmmm have any of you heard about Kiwi. It has not been released yet, but is undergoing development. More info here http://www.theronge.com/kiwi/ and here http://www.theronge.com/

Kiwi was mentioned in a few comments here (including one from Matt Ronge) and more recently on Hawk Wings:

http://www.hawkwings.net/2006/06/15/kiwi-new-imap-e-mail-client-in-the-works/

Some consider Mulberry the “gold standard” of IMAP clients. I used it for +5 years before switching to Mail a couple months after Cyrusoft filed for bankruptcy last October(?) and never lost even a single message that I’m aware of.

I too am a longtime Mulberry user. I still use it as I didn’t understand the need to jump ship just cause the company filed for bankruptcy. The program still works and it meets my needs better than the other IMAP clients I’ve tried in the last year.

Not so much individual needs, but from a large scale IT operations perspective, one feature is sorely needed.

1) users should be able to define a workgroup - a circle of people that share access to a common repository. Attachments should then only move the link to people within the workgroup, allowing shared access to that file. People outside the workgroup would receive the file.

This would significantly reduce bandwidth requirements over the network, and reduce problems with configuration management (revisions) of the data.

Eudora does this, to some degree, but is a difficult sell in the corporation.

2) Connectivity to Exchange - even if this product is a dog, it is prolific in the corporate environment, and integration needs to exist.

20 June 2006

by voline

I want an email app to store the mail in an open, well-documented format, preferably MailDir or mbox. I would also like smooth integration with GnuPG encryption.

Another feature I’d love to see is one I got used to in Eudora, but can’t find in Mail.app:

Creating a folder on-the-fly, either while executing a Move To, Copy To or while creating a rule which uses same.

@Chris re: automatic mail checking and SMTP servers (Comment 1059)

You can turn off auto checking for each account you have configured. It’s in the Advanced tab under Accounts.

You can also have different SMTP servers. But, I wonder if I understand your problem correctly, because with the many varied email accounts I’ve used, the SMTP server for that account worked regardless of where I was connected from: home, work, Starbucks, library, airport terminal, Ukraine, Rep. of Georgia, Russia, Germany, etc. It’s the rare occasion that I need to worry about SMTP servers and that’s only when the connection is down.

I too am a longtime Mulberry user. I still use it as I didn’t understand the need to jump ship just cause the company filed for bankruptcy.

Mulberry’s back and now available for free, as you probably know by now.

hi.

i was complaining on some apple forum a while back about how badly threading sucks. the tech that works there told me they actually do so-called ‘real threading’ since panther or something, based on the References header.

he also told me that this problem, like many problems, comes from users behaving badly. instead of initiating new messages (which References: ), many people compose e-mails by replying to old messages from the desired recipient, and manually clearing out all of the fields.

the result is a totally unrelated message showing up as part of the same thread.

i don’t see an obvious resolution for this, except maybe imposing the restriction that threads must have the same subject or ‘Re: ‘ + the same subject as the original subject.

just some thoughts. if anyone could write a mail application to slaughter mail.app, it would be macromates. but i hope that you write remote editing for textmate first :-)

-N

I am not a mac user, but a Unix/Linux user, and I use Thunderbird at work and mutt at home. If I were convinced that mutt would work as well over IMAP I’d do that.

Mutt is infinitely configurable, much like EMACS. And while I’m sure being a text-mode interface would not match the Mac GUI well (Apple does user interfaces wonderfully), it is very responsive, and if you “do” email a lot during the day I’m sure those fractions of a second add up. Check it out, there’s tons of info on the web about it… You can use fetchmail to download your IMAP email, process it with procmail, deliver it to a Maildir folder, and then use mutt to read it and respond.

17 January 2007

by Leif Halvard Silli

What you forget – or do not mention – is writing.

I use Thunderbird, because it is the best e-mail application for writing e-mail with - in my view.