Can Mail.app cope with heavy loads?
Dave Hamilton from the Mac Observer is thinking about switching to Mail.app.
He has had a gutful 
of his old mail client:
Mailsmith, by most rights, has stagnated. It pains me to write this, because as I said, I’m a really big fan of BareBones, and enjoy a great working relationship with their head-honcho, Rich Siegel. But it’s true… Mailsmith hasn’t had a public release/update since March of 2005. Now some folks may argue that it doesn’t need an update, and for those folks, I’m sure that’s correct. My big problem is that I manage a LOT of e-mail… I have almost 1400 mailboxes within which are nearly 200,000 e-mail messages. I pretty much save everything, and it’s saved my ass in HUGE ways over the years, so I ain’t gonna stop.
So, he wonders, does Mail.app have what it takes to manage a large number of mailboxes and bucketloads of email? Or is he better off archiving off a large slab of the emails and staying with Mailsmith?
My advice is not much good. I only have about 35,000 emails spread over four IMAP accounts and about ten mailboxes (see further, “How the delete key is your best friend”). That’s chicken feed by Dave’s standards.
Justin Blanton once ran a challenge to find the largest Mail mailbox, putting up his own inbox of 22,000 as a candidate.
What’s your experience? What’s your biggest mailbox? How many mailboxes does your Mail.app handle without working up a sweat?
Can Mail.app take the load? Does size matter?
Tags: Apple Mail, biggest mailbox, Email in general, email volume, mail.app, mailboxes, Mailsmith, switchingRelated posts
August 21st, 2006 at 11:58 pm
Thanks for posting this, Tim! Just a clarification — It’s not so much the size of any one given mailbox that’s my issue… just the opposite, in fact. I have a lot of messages (200k) archived across some 1400 individual mailboxes (”On my Mac”, as it were). I tend to file pretty religiously, and that’s one of the things I like about Mailsmith — I can do all my filing with keyboard commands and by typing the first few characters of the intended destination mailbox.
In any event, I look forward to hearing what any of you might have to say!
August 22nd, 2006 at 12:14 am
At one point I filled my email in a similar way to Dave, I had around 100 mailboxes and somewhere between 50,000 and 55,000 email messages. I had the same philosophy as Dave, if I kept everything than I wouldn’t have to worry about things, I could prove whatever because I had the records. After switching to Thunderbird for a few weeks (for other reasons) and disliking the experience I switched back to Mail.app and realized that I had far too much email and then I started deleting alot of it and now I’m down to about 1600 messages in 15 mailboxes, all of which is stored on IMAP now (one of the other reasons I deleted alot, it took up far too much space).
I know my old setup is nowhere near the size of Dave’s but it’s something.
August 22nd, 2006 at 12:26 am
Hi,
I just moved my mail from Outlook 2003 to Mail.app and so far so good :).
I have around 150 000 + messeages in 100 mailboxes and Mail.app works like it should. There is one thing and that is when u click on mailbox that holds 30000 messeages ii allways starts to index or something. I can se a little circle and than i have to wait 15 - 30 secs before my messeages shows. I’m not sure why is this happening but it does. It doesn’t affect my work but it is there :)
August 22nd, 2006 at 1:04 am
I don’t file everything religiously, I do keep everything of significance in an “Archive” mailbox. All archived items must have Mail Tags keywords attached, (otherwise the “Act-on” rules that I apply won’t move the e-mail)
Thats about 2500 messages
I have mailing lists that are sorted into mailboxes and items are held for a bit over 6 months, that adds up to as many as 4000 emails for these lists, depending on traffic.
I receive hundreds of e-mails a day. Most of them spam (I have a knack for getting on the weirdest mailing lists on earth, even with snail mail).
The mail.app spam filter is okay for the average joe who’s been careful with his e-mail address. But if you have a lot of spam, SpamSieve is better. Though it needs a bit of help early on, over the long haul it’s extremely good at filtering spam)
Handling many types of e-mail is aided by “Mail Act-On” and “Mail Tags”, which I’ve mentioned earlier. These two mail.app plug-ins turn mail-app from a basic e-mail program into a power user mail program.
August 22nd, 2006 at 1:47 am
I have about 110k mails in Mail.app and it works. It’s not fast, but compared to Mailsmith it flies.
I have two mailboxes with more then 20k mails, if I had one of them in Mailsmith it would have broken my entire workflow because Mailsmith was unusable with mailboxes containing more then 3k mails . That’s why I moved to Mail.
August 22nd, 2006 at 1:57 am
Thanks, Guido and all. Yeah, my big problem with Mailsmith isn’t so much that I have a lot of messages in one mailbox, it’s that I have lots of mailboxes (1400) and, for whatever reason, having that many mailboxes seems to bloat up Mailsmith’s RAM requirements to the point where it bogs down the rest of my apps (and I have 2GB of RAM in my machine!). But after reading Guido’s message, it sounds like I’m doomed either way… even if I did archive everything down to a handful of mailboxes, I’d have the same problems that Guido’s having. Hrm…
August 22nd, 2006 at 2:09 am
“I have lots of mailboxes (1400)”
Why?
It sounds like soring email into that many boxes can’t serve any useful purpose (other than an obsessive / compulsive disorder).
In fact, I’d bet that it takes a lot of your time to manage them, never mind the ability of programs to handle so many boxes.
Better to file stuff away in fewer folders in broader categories, and trust in Spotlight to search for emails.
August 22nd, 2006 at 2:12 am
Does any mail client work well with this many emails? There is obviously a need for an industrial strength mail client, especially as time goes by and more people begin to depend on their increasingly large email archives.
Maybe Apple needs to create a Pro Mail client (something to Mail like Aperture is to iPhoto)?
August 22nd, 2006 at 2:25 am
Shawn and Jeff ask two very good questions, so I’ll try to answer them as best I can.
For Shawn — the reason why I do this is because I started doing this in 1994, when the searchability of e-mail clients stunk. This was the best way to keep things organized back then. I currently run/manage 4 businesses (and have had 2 others in the mix there), and in keeping things organized and separate, well, the number of mailboxes bloated up. Even now, with search capabilities being much better, there are still times when it’s handy to simply dive into one of these boxes to find something. That said, I do appreciate that technology has changed, and perhaps having all of my e-mail in one monster archive folder isn’t necessarily the wrong way for me to go. Heck, a lot of my searching is done with Gmail now, and that’s exactly how my mail is stored there.
Jeff — I don’t know the answer to that question. From what I had understood, Mailsmith did fine with this many e-mails, it was the amount of e-mail *boxes* that was causing my problem. Mail.app and Eudora did fine with that number of boxes, and Eudora handled the mail just fine, too. I just got sick of the klunky Eudora interface and, like I said in my blog, at the time I figured it was worth my while to experiment with Apple’s (then new) Mail.app. But now I’m hearing mixed reports — some say that Mail and Mailsmith can easily handle this many messages (and more), while others indicate exactly the opposite. Obviously machine resources, processor and harddrive speed, RAM, etc., all factor in here.
Still… I appreciate these comments and all the others. This is becoming a valuable discussion, not just for me, but as an archive for anyone looking to make a similar decision down the road.
-Dave
August 22nd, 2006 at 2:29 am
Dave,
Did you try filing a bug report with the MailSmith developers?
August 22nd, 2006 at 2:51 am
Mulberry is now free. It’s not the prettiest or easiest to use client, but it’s almost infinitely configurable and intentionally standards-compliant. It’s second to none for IMAP, too, which is why a lot of universities distribute it as their preferred client. Why not give it a whirl?
http://www.mulberrymail.com/
http://en.wikipedia.org/wiki/Mulberry_(email_client)
August 22nd, 2006 at 2:58 am
I have just under 240,000 emails spread across 81 folders. The largest of these has just over 73,000 emails. These folders are on my IMAP server, and are synchronised locally on my Mac.
I’m used to it taking up to a minute to open the larger folders. Opening a message is usually instant, although if Mail is busy with other IMAP stuff in the background it can take 10 or 20 seconds.
However, most of this delay seems to be due to communication with the IMAP server. If I’m on a dial-up connection it is dead slow; when I’m on the same network as my mail server, it’s pretty speedy.
Moving messages between folders is basically instant as far as the UI goes, although the IMAP operation in the background takes longer. Having said that, I sometimes get an issue where I switch away too quickly from a folder that I’ve just moved a message out of, and it doesn’t complete the move.
Spotlight searches are satisfyingly quick. I can usually search my entire archive within a minute and a half, and I can search an average-sized folder (say, a few thousand messages) in seconds.
Most of my slowness seems to come from the fact that it’s syncing everything with the IMAP server all of the time. I get the impression that the local store is actually pretty efficient.
August 22nd, 2006 at 5:21 am
I have around 20,000 messages in 5 IMAP accounts, with and additional 8 shared IMAP boxes inside those.
Take about 10-20 seconds to load the inboxes equally fast to navigate or move between boxes.
The only area that could use improvement is Spotlight searches can be time consuming and inaccurate if you use other systems outside mail to manipulate the contents.
It requires a “rebuild” of the account or folder to get the search back in sync.
All in all. It rocks!
No other e-mail client out there provides the unified inbox UI that I can find. This save SO much time in my opinion.
August 22nd, 2006 at 5:50 am
I sometimes see my dad checking his email, and it looks like he has close to 175,000 emails in his inbox, and more being added everyday. Be it that he doesn’t like to delete them much, the size of his mailbox is constantly growing. (He uses Mail.app) I have never seen him complain about lag inside of mail.app, everything loads and opens instantly. Except, obviously, for the huge emails with about 12 images. I think the only thing that mail suffers from is the fact that it’s slow to communicate with servers. Other than that however, it manages big mail perfectly.
August 22nd, 2006 at 6:01 am
I also have over 100 inboxes, and over 150,000 messages, collected over the last 12 years. It has also saved MY ass many times to have that archive. I will not stop either.
I can answer your question very simply; No. Apple’s Mail cannot cope with large amounts of email. I tried switching to it twice, and it was…how shall I put it mildly…”painful”.
There is only one mail client I know of that can handle what I have, and that’s Eudora. Unfortunately, Eudora has been under development for a “modern” OS X version for like 3 years now. Qualcomm claims someone is working on it, but there has been absolutely NO visible progress. It’s a tough situation, because Eudora offers no Unicode support, shitty HTML support, and a whole bunch of other problems. But, searching through literally hundreds of thousands of emails happens in like 1-2 seconds on my MacBook, even though it’s old as hell, and running under Rosetta.
Anyway, that’s my two cents. Every year or so I evaluate the mail client market, and consider a switch, but I never have.
August 22nd, 2006 at 6:07 am
Marc — I think you might have answered Shawn’s question of “why?” better than I did… As I indicated I, too, started with Eudora, and left for all the reasons you mentioned. However, my current mail-storage schema was set up within Eudora, and I’ve never been able to un-learn myself from that technique. Eudora could handle it, Mailsmith and (at least when I last tried) Mail couldn’t. That said, some of the comments here have indicated that the current release of Mail might just be good enough (perhaps not as good as Eudora, but better than Mailsmith)…
August 22nd, 2006 at 6:33 am
Well, my email storage schema is certainly not what I would call ‘obsessive’. I have ~100 or so inboxes, because I have things organized by project, and I have had about that many projects over the years.
Sometimes I am working with the same people but on different projects, and if one of those people sends me an email like “the image files are on the server”, how the hell is spotlight going to find that, and further, know that it is attached to a particular project? On a different note, Spotlight’s mds/mdimport processes are a major resource hog, and as a search engine it basically sucks (no simple filename search, crap/no boolean options, etc.).
Anyway, I don’t see how multiple inboxes counts as some kind of arcane ‘technique’ that Eudora trained me for? I have my files organized by project also, in the file system. The Finder seems to cope just fine with the hundreds of thousands of folders I have, and literally millions of files I have on my machine. So why the hell can’t Mail handle 1/10th or 1/100th of that?
I would love to take advantage, for example, of Mail’s ’smart folders’ and stuff, but I know that if I dive in and add those, I’m looking at even *worse* performance than I saw when I just imported the mail and clicked around a bit and set up some filters.
Someone else in the thread says that when they click to view an email in a large inbox, they’re looking at 10-30 second wait times to see it. That’s just not acceptable for me…calling up a plain (or even Unicode) text file should happen in less than a second, at all times, unless you are like crunching a FCP video and playing WoW simultaneously in the background while Spotlight indexes your hard drive, or something.
I’m not interested in IMAP, because I want all of my email to be completely offline. Mail stored on an IMAP server could be stolen by any reasonably skilled hacker, and my email contains sensitive business information, as well as personal stuff that I don’t want out in the open because of some kind of as-yet-unknown server exploit.
Sorry if my comments imply a tone toward the others here. Really, the absolute shit state of email programs on OS X is what causes me this stress. :)
August 22nd, 2006 at 6:49 am
OS X Mail App is not known for its stability or speed when dealing with large amounts of Mail. By large amounts of Mail, I mean 5,000 plus and if you get to 20,000 plus mail messages, heaven help you.
Now this just may be related to (woeful) and somewhat extensive experience of Mail App with (Exchange 5.5 Mail servers) and it may not JUST be Mail App, it may be that the Mail App “Sucktitude” is the Combination of Mail App and Exchange 5.5, not Surprisingly, when Exchange 2003 goes in soon, we’ll see if things improve, I think they will.
I have no issues with Mail App and .Mac, (which Apple is using OS X Server Mail) [IMAP] and it is fine, been fine for years.
I recommend in a big way, Mail Steward, it is great, can Archive 100 of thousands of emails (SQL type database) and you can Spotlight search, etc., get them out of Mail app, kind Offline them, but still can search / find mail. Hope they build this Archiving power into Leopard Mail, we’ll see, it is crying for it.
In my environment, the performance and stability of Mail App, is directly related to How many mails one has, if I see 10,000 mail messages and people are complaining of Mail App issues, I say: dude, Clean Out (Archive in Mail Steward) your Fuckin’ Mail Box (of 7 years of mail), there is too much stuff in the damn pipe, that usually does the trick. Ah, now, that felt good, to get off my chest.
August 22nd, 2006 at 7:05 am
My inbox is approaching 10k (9,751). I am using IMAP only, my largest mail folder (stored server-side, mirrored locally) has 33,823 messages. Spotlight is able to search the entire folder almost instantly. In total, Mail.app handles about 60 messages per day, and stores over 100k messages, without hesitation.
August 22nd, 2006 at 9:22 am
There’s a paradigm problem here.
I don’t need my day-to-day email client to handle 10 years of email archives. Instead, I’d love to separate and archive my older mail and use a purpose-built archival mail app to access it. One with faster, better search and threading tools. Those tools exist, but are still pretty immature and unsatisfying. But they could be made much, much better. In fact, I’d venture that Bare Bones is just the kind of company to build such an ARCHIVAL mail app.
August 22nd, 2006 at 9:57 am
I don’t need my day-to-day email client to handle 10 years of email archives. Instead, I’d love to separate and archive my older mail and use a purpose-built archival mail app to access it
As I said, Mail Steward
August 22nd, 2006 at 11:44 am
I’m of the opposite view: Personally, I don’t want to spend $50 on an extra app that’s outside my email client simply to archive *text files*.
All Mail Steward does, basically, is read in your text files, put them in a SQL database, and allow you to recall them. Woo-hoo. Explain to me why that’s worth $50?
(I confess to not being a user of that program, but as an aside: the comments on versiontracker.com are, needless to say, not inspiring, and neither is the author’s demo limitation of 3,000 emails (i.e., why is he afraid to let us see performance with 100,000 emails; why not a timeout limitation?).
Here’s my idea: if Mail.app (or something else) was simply written efficiently, Mail Steward would have no market, and we could all save our $50. I don’t want to screw with my workflow because some programmers are lazy, or because some marketing people don’t want to give a free program the resources it deserves as part of the OS.
In fact, my problem would simply be solved by an Intel-native Eudora, but Qualcomm is dragging their heels as well…there’s an opening in the OS X market for a decent email program, but I think the mere existence of Mail has killed developers’ desire to compete.
August 22nd, 2006 at 1:11 pm
I have 22K+ messages (750+ MB) in one IMAP account distributed amongst about 20 mailboxes, plus a dozen smart mailboxes and ten filters. Only about 1600 messages currently reside unfiled in the Inbox itself. On a variety of contemporary G5 and Intel Mac hardware running 10.4.7, the speed is more than acceptable. Only opening individual mailboxes for the first time in a session seems to impart any delay. Opening individual messages is instantanenous. I access the account over a fast DSL connection but have not tried it over dial-up. First access from a new computer can take a long while to download so many messages, as I don’t just download headers but the entire message contents. This allows full access to all but the latest messages while traveling.
I really appreciate being able to access everything from any of my computers, but like others have expressed, I do not like the potential for lack of privacy at the server side.
August 22nd, 2006 at 2:04 pm
Unfortunately I believe the answer is no for many mail messages. The reason being that each message is stored as a separate file. Because of this loading mailboxes are very, very slow. Of couse I only have a Dual G5 @ 2.0Ghz with 2Gig of memory, but the load up time is very time consuming.
I need to investigate archiving software.
August 22nd, 2006 at 3:00 pm
Aron, what version of Mac OS X and Mail are you using? All of my Intel Macs have “only” 2 GB and, as mentioned above, Mail performs with 22K+ messages just fine.
August 22nd, 2006 at 4:33 pm
I have almost every (non-spam) email that I’ve sent or received since 1998 and a few from 1997. It’s about 3 GB of email. All in IMAP, and all in Mail. I have it set to cache all messages, omitting attachments, and Mail seems pretty darn efficient, once it’s done caching everything. I just wish it wasn’t so brain dead when it comes to searching.
August 23rd, 2006 at 12:08 am
Just did a quick check, 45 mailboxes, 113k messages (19K unread! I blame that on heavy traffic mailing lists)
My one experience about Apple Mail that is appropriate here is that I’m using an Intel iMac, and my previous iMac was a G5. The difference between Mail’s performance with this many messages is DEFINITELY noticeable. Re-indexing, even in the largest mailboxes (15K messages) is practically instant - something that used to be painful on my G5 iMac. So a big thanks to the programmers that tweaked Mail’s performance on the Intel side. And to Intel itself obviously!
August 23rd, 2006 at 12:32 am
I doubt Apple has tweaked Mail for Intel. Apple’s 32-bit Intel systems are just nearly as fast as the latest PowerMac G5 systems and way faster than the single-processor G5 iMacs.
August 23rd, 2006 at 8:30 am
Coming from Eudora/Netscape Mail on Mac OS 9, I decided to fully commit to Mac OS X since the public beta release I got in Paris on sept 2000.
Since the I used Mail.app only in IMAP with local cache. I enabled caching only since Tiger in order to get Spotlight to work.
Mail worked like a charm. Sometime slow in all mailboxes-wide searches but not noticeables.
I cannot cut and paste the account info window but roughly in mail work account there are 80-100 IMAP mailboxes ranging fom 30000 (one) to 10000 (several) to 1000 (many) to 100 (dozens) message counts. I estimate about 100000 messages. Some messages carry attachments of 5-10 Megs.
I notice no difference from year to year: Mail.app seems able to manage a countless number of messages with no slowdowns.
The only feature I miss is true IMAP search, not using local cache. If I perform a full text IMAP search in Thunderbird or in our webmail the results come istantly. Using Mail + Spotlight I have to wait just some seconds.
August 23rd, 2006 at 12:05 pm
Well, this has gotten interesting. Unless I’m really missing something, it appears as though Mail.app doesn’t export its mail out in a hierarchy, either. Is it impossible to tell Mail.app to take your mailbox hierarchy and simply export it to a series of folders and “real” mbox files?
August 23rd, 2006 at 12:36 pm
I have a “smart” folder setup, mail said approx 100,000 messages. It took 25 seconds to start displaying and it’s been going on for minutes now to open the mailbox. You can see it in the activity viewer. Mail is very slow the first launch of every day.
Think about it. All the messages are separate files. For Mail to display a changed folder, it has to go through the cache or at worst, index through all the messages. They are separate files. This is a very inefficient way to handle large numbers of messages. Any software engineer will tell you that this is possibly the worst way to store messages from a speed standpoint. It’s been minutes now and it’s still updating the “smart folder”.
August 23rd, 2006 at 6:00 pm
Aron, you’re right: I forgot in my comment (it was late at night in Italy…) to mention that at the first lauch of the day Mail.app is quite slow.
Nevertheless it becomes more usable after several seconds: you don’t have to wait for the cache refresh to be complete in order to send a new mail o read new mails.
I’m used to keep Mail.app open all the times: I’m on a PowerBook and I rarely restart it.
I noticed also that relaunching Mail.app after the first daily launch takes less time.
August 23rd, 2006 at 8:51 pm
i’ve also kept every single email since 1995, used nearly every email-client available over the years but sticked with outlook (from version 97 onwards. last year i decided (with no special reason, just to see something different) to switch to a mac. my experience after having imported 9x 2gb .pst-files from outlook (2003) via the littlemachines-tool (which was a pain in the ass for itself) i noticed, mail was (in comparison to outlook) unusable (for me) for 2 reasons: general speed (e.g. 3-4 secs of waiting when clicking “new mail” until the window finally opens, waiting several seconds when switching from folder to folder, as already stated above), but, most importantly the miserable search speed (also in comparison to outlook with the lookout plugin). i used to do a fulltext search on, say, a 2gb folder with 40.000 messages within 3-4 seconds, and a fulltext search on the whole 18gb pst-files with ca 500.000 mails within max 25 secs. in mail.app i had the subjective feeling of waiting the factor of 2-3 times longer, and the search results hadn’t been as good/precise (tested it with similar searches on both programs, probaby spotlight lookout).. i’ll give mail.app a next try when version 3 becomes available, although i think they won’t be changing things of this kind (more efficient caching, changes in data storage or whatever, but, if the rumored ZFS filesystem really comes, this could be a performance boost… what do the experts say?).. i stick with entourage for the moment (which is not better as well ;).
i’d be very interested in a permanent email archiving solution, like someone posted above, as well. i think one could get rid of emails , say older than 6 years in mail.app directly, if the messages remain searchable.
which other usable software despite mailsteward is available at the moment?
one big criteria for me would be platform independency (at least for the frontend, as i work on win, mac & linux) and storage in an industry format or at least the possibility to export the stuff again in mbox oder eml or whatever.. i would’nt mind if it was server based with web-frontend or whatever..is there anything on the market, best licensed as open source?
excuse my bad english
August 24th, 2006 at 12:34 am
[...] If you’ve been following the conversation here (or over on HawkWings), you’ll note that one of my biggest concerns with having all my mail *in* Mailsmith was that I couldn’t easily get it all *out*, especially not while preserving the hierarchy of messages. That’s no longer a concern, thanks to a little bit of AppleScript mojo, a few hours of “free” time, and the seeds planted by BareBones flat-export script. With great fanfare, but clearly no guarantees, representations, or warrantees by the author or anyone else, I present to you my Mailsmith Hierarchical Export Script. [...]
August 25th, 2006 at 11:30 am
Sorry, people…I really don’t mean to offend or insult anyone — but in my eyes collecting each anyd every mail over years or even decades and keeping hundreds of thousands of mails in hundreds or even thousands of mailboxes/folders is just nuts and obsessive…!?! It doesn’t make any sense at all…! Even more so with people saying that they have like 19K unread mails…!? When are you going to read those anyway…?! They just keep piling up, that’s all…
Old mails saved your a$$ so many times…?! I just can’t think of any example where an email from 1996 could save my butt today…!?!
I guess some people feel better collecting and storing all kinds of stuff or information and make up resons for doing so…just like that weird old lady down the road who had to move into her own garage, because she didn’t have space for herself in her own house which was filled with newspapers and magazines from the last 60 years or so…
August 25th, 2006 at 11:37 am
mangaphreak - Like you said, you can’t understand why people would keep old e-mails. There are those of us (myself definitely included) who can’t understand why people *wouldn’t*. And the good thing is, there’s no right or wrong here… whatever works for you, well, obviously works for you, or you would change it. The same is true for me and, like I indicated on my blog, having e-mails from years and years ago *has* saved my butt (and that of the various companies I’ve owned and managed) more times than is necessary for me to justify keeping everything. :-)
It’s my way, is all… not right, not wrong, just mine.
August 25th, 2006 at 11:46 am
This alone has been a savior (for desktops , on the network, not going anywhere.
Mail, Preferences, Advanced, Dont Keep Any Copies, (which is a kind of Apple’s stupid way of saying, “Download Subject Headers Only”
Activity View is very fast. However, this does excuse the faxct that Apple, must create a damn better, faster and more stable mechanism for storing / retreiving mail.
Another peeve, once your local mail is “downloaded, it is “Spotlighted”, metadata, as emlx files.
Guess What, it looks like even Mac OS X Mail App can not Import, emlx mailboxes, you must convert them to mboxes and then import them.
The whole damn thing (Mail) needs to be re-written for speed, stability , performance and interoperability (Exchange), etc.
August 25th, 2006 at 12:36 pm
Since Manga brought up my example of 19k unread messages, I figure I should respond. The 19k is (as I quickly explained) from some high-traffic mailing lists that center around tech support questions (I’ve actually *read* about 8K of those mailing list posts over the last 4-5 years). Whenever some obscure tech issue crosses my desk, I’ll search those lists for possible clues if not outright solutions. It has served me very well in the past. And also, as Dave Hamilton comments, having at my fingertips the entire history of my email ‘life’ has been handy when having to recollect some distant long ago issue or correspondence with someone.
I’ll happily archive it all when it truly gets unwieldy, but right not Mail.app is handling it just fine, and plus none of the archive solutions out there really fit well with how I like to do things day to day.
And I’d like to close by reiterating that I think going from PPC to Intel, somehow, Mail.app improved by quite a bit. I went from a 1.8ghz G5 (10.4.x) to a 2.0 dual core Intel (10.4.x), both with over 1GB of RAM, and the difference between Mail’s performance of certain functions on the two machines was like night and day. More than what I would expect from going single to dual core. So, once again kudos to Apple.
- M
August 25th, 2006 at 2:04 pm
The main reason I started keeping all my old email is it’s like an awesome autobiography that I don’t have to write. It just magically happens. I can look back a year or two and see what business decisions I was really concerned with, or what was going on socially. Tiny stuff that I otherwise would have completely forgotten about.
In fact, once I crack open the archives, I tend to get absorbed into reading them for hours, so I try to avoid it, and am saving them for when I’m older and have nothing better to do with my time than reminisce. :)
I consider it different from your example of hoarding old newspapers because:
1. It’s information that is (or was) directly pertinent to me.
and
2. It’s 3 gigs. That’s like 1% of my available hard drive space and falling. Maybe 1/8th of the size of my music collection. Storage is cheap. Why not keep it?
August 25th, 2006 at 2:52 pm
I totally agree with stevenf about the autobiographical nature of mail archives and I use my mail archive in a similar way.
In Mail (I really don’t like to call it Mail.app) I have over 200,000 messages and my ~Library/Mail folder is 5.1GB in size. These are messages going back to about 1998.
Performance in Mail.app is fine as long as you don’t have too many emails in a single mailbox. I find that once you get more than about 15-20,000 emails in a box it bogs down and I usually create an “archive” mailbox to shove the old mail into to speed things up.
Searching is reasonably quick even in mailboxes with 20k+ of messages. It is certainly MUCH quicker than the last version of Entourage that I used (Office v.X) which was simply painful (and the main reason I switched to Mail).
I use SpamSieve to handle spam filtering which works great, the built-in junk filter in Mail is pretty crap.
August 25th, 2006 at 4:15 pm
Re: the transition from a single G5 to a dual-core Intel, those home version G5 systems were intentionally crippled. Slower processors, little L2 cache, memory-bus running at 1/3 speed. Then come the Core Duos with big L2 caches running at full speed, memory bus not crippled. Oh, and 2 cores each using clock cycles way more efficiently than the G5 (which was significantly less efficient with cycles than the processor it replaced, the G4, even in PowerMacs).
August 25th, 2006 at 4:57 pm
About having large (20k+) archive or legacy mailboxes, a feature I’d like to see in Mail.app is mailbox subscription (a standard IMAP feature): on Thunderbird (or at least on old Mozilla mail) once can decide which mailbox subscribe or not in order to keep syncing and searching faster from inside the mail client.
August 25th, 2006 at 7:00 pm
I don’t trust local mail clients, so I tend to archive my mail on the IMAP server, and having a ~15K messages inbox (most of it spam that I haven’t filtered yet) doesn’t help. Thunderbird on linux and windows can deal fine with this setup, but on OSX it’s painfully slow, as is Mail.
So I’ve begun to move my mail to a locally installed IMAP server, which helps a bit. But it’s still a pain to move mail from my ISP to the local mail server…
Now that I’ve read this thread, perhaps if I leave Mail open enough time it will catch up with its indexes and not be so painfully slow, I’ll have to try it
August 25th, 2006 at 10:57 pm
I’m amazed and fascinated at the numbers quoted: 10 years of email, quarter of a million messages,etc. But then I have 2GB RAM and 1TB of hard disk space which I can’t imagine ever filling up. Naive? Probably, but compared to my previous Mac it’s cavernous. But as stevenf points out, it’s less than 1%. So nowadays we keep more things simply because we can.
Dave Hamilton, I think, is a self-fulfilling prophecy: as the amount of information kept grows then the butt-saving chance increases. Be wise - archive!
Thanks to everyone for making me think about this before my emails number in the thousands. From now on I’ll be asking another important question when I receive email: can I be 100% certain that I will never need to refer to this again? If there’s any doubt I’ll keep it.
It is also blindingly obvious that MailTags is essential.
August 27th, 2006 at 9:14 am
Well, as I said before — I didn’t mean to offend or insult anyone…! And of course there is no wrong or right, everyone can do with his mails what he (or she) wants…! :)
I still consider it a bit weird and strange though…and ..getting back to the initial topic — I also think that none of the current mail apps are actually built for handling such amounts of messages…!? Maybe some apps should have like a Pro version that can cope with such — still rather unusually large — numbers of mails and directories…?!
September 2nd, 2006 at 5:02 am
[...] As many of you know, I’ve been in flux with my e-mail management system, and I think I’ve found a very workable solution for me. Though I discussed it on the most recent Mac Geek Gab podcast, enough of you have e-mailed me about this that I thought I’d type about it here, too. [...]
September 12th, 2006 at 4:15 am
[...] Per chi è vissuto di Internet negli ultimi anni la mailbox diventa un archivio permanente, anzi una vera autobiografia come ha detto Steven Frank (poi approfondito nel suo blog). [...]
January 11th, 2007 at 12:29 pm
Victor wrote:
>”I don’t trust local mail clients, so I tend to archive my mail on the IMAP server…”
I thought that comment was interesting. I’ve been interested in trying IMAP more, because I use multiple computers, but was worried about trusting the server.
If the server accidentally wipes out my email and I connect and “synch” then my email locally gets wiped out too, right?
doug
January 12th, 2007 at 3:08 am
If the server accidentally wipes out my email and I connect and “synch†then my email locally gets wiped out too, right?
No, not if you have Apple Mail (in its default state /settings) it is bringing down your mail locally (yes, Cached). So /Users/’username’/LIbrary/Mail
Copy / backup this folder and you are good to go.
But IMAP is the way to go, and this new push-IMAP I can’t wait to see (with regards to the iPhone and Yahoo)
Anyone care to specualte where Apple is going with this new Yahoo (push Mail), very interesting.
What does this mean for .Mac, where is Apple going with this Yahoo connection, very interesting.
January 12th, 2007 at 3:18 am
My own follow up, clarification on IMAP and Apple Mail
In Apple Mail’s default state (in IMAP configuration) one’s mail is on the server (so mail is acting like a viewer) you just view your email on the server and it STAYS on the server (the beauty of IMAP) assuming your settings are set to and they are by default in Apple Mail.
But, Apple Mail also “caches” your mail locally into the above location I stated above, this is useful for laptop users (if they use a laptop for example at their work and bring it home, they will be able to view/read their email since it is cached, Apple calls this, “Keep copies for offline viewing” in the Advance tab in Mail App.
If you have multiple computers (and these days many of us do, heck even if you do not) IMAP is the way to go, it is backed up (depending on who you use (.Mac). I have been using .Mac since its inception and have never been without an email connection or lost data, (it is backed up).
Of course you still have your local “cached” mail you can go to as a safety measure.
January 12th, 2007 at 3:45 am
Incidentally,
I am an Apple engineer / admin at a hospital in Los Angeles, about 120 plus Macs and we just upgraded to Exchange 2003 (finally, thank heavens) and we are using the Exchange setting in Apple Mail.
It works very well, as I anticipated, since the Exchange engineers did a type of clustering to maximize email performance, etc.
It is not uncommon for a user to have 5000 plus emails and a few have 13,000 emails. They now have a quota of about 1 GB to 2 GB of space, (keep in mind attachments in your inbox and sent will count against your space quota).
E-mail is coming down (caching) pretty quickly now, even on the initial setup with 5000 plus emails, the cache download is about 15 minutes, if that.
On the Exchange 5.5 / NT 4 servers, this would take hours and more often crash Apple Mail.
People were mistakenly thinking Apple Mail app was horrible, but it was the Exchange 5.5 / NT 4 servers (of course we the engineers knew this), heck even the Windows Outlook clients were slow and crashing.
Apple Mail is pretty solid and getting better.
New servers, all is great now.
Another thing on Exchange Servers and Apple Mail, Public Folders.
A nightmare for Apple Mail app connecting to Exchange (2003, via IMAP / Exchange).
If your site has tons of items in Public Folders (and we have 51,000), Apple Mail just interprets these as Mail messages and wants to cache /dowload them (Apple Mail default). (Yes you can use an IMAP Prefix Path, but that can also block you from seeing other folders, etc).
This sucks, since now you are congesting Mail app unnecessarily with traffic and e-mails you can care less about, and it is constant activity / syncing.
Solution: I sent an email to our Exchange admin with all of my Mac users (this will eventually be handled by Active Directory).
He will make a script to “Exclude” these Mac Mail App users from seeing Public Folders.
Can’t wait for Leopard Mail 3.0, should be great
Anyone who wishes to correspond on Apple Mail and especially in an Enterprise / MS Exchange 2003 environment, feel free to email me, I am thinking of getting a Blog / correspondence on this, it is huge and all of us Mac users, Mac / Windows admins could use the knowledge.
macguitarman@mac.com
November 18th, 2007 at 7:30 pm
Hello All,
I have the following problem I have a 181000 mail inbox on apple mail and it crashes every other second. I’m using Apple mail 3.01, i rebuilt the envelop etc. The total mail folder is about half million mail for about 12G.
my inbox is a way over 2G.
Entourage works but is damn slow. Can anybody help Me?
November 22nd, 2007 at 8:14 pm
My current favorite setup is to use Gmail’s IMAP as my repository of email, and to sync it using Leopard Mail 3.0.
I am very careful to not allow mailboxes each to get too large, hence, I archive according to year, e.g. 2007, 2006, 2005 etc. for general emails.
For specific issues: letters, business projects, each of these are in separate nested folders and subfolders, hence, no one mail box gets too large.
I have emails going back to 1996, and I’ve kept almost everything.
For people who don’t think email archives are important — it occurs very rarely, but sometimes you need to find the contact of someone, or some small piece of information. That’s where a lifetime email archive comes in. I am in the legal profession, and I find it imperative to keep email archives of business correspondence, in case a client refutes something. You can then dig into your email archive, and prove it in writing. Nothing beats keep records. People who don’t bother keeping email records don’t need this level of detail in their life, but there are those of us who do.
I’m very happy with Gmail IMAP synching with Leopard mail.
I can’t comment on size limits because I have not pushed the system to its limits yet. It is purring along nicely.
The advantage of using Gmail IMAP is that you can use Google Desktop search to index the gmail archive. If Spotlight can’t find it, maybe Google search can.
With my setup, Spotlight indexes the IMAP emails on my Mac, while Google Desktop indexes the same IMAP emails on the Gmail server.
Giovanni Pau @ Nov 18th — can’t you split your 181,000 into several subfolders, e.g. 2007, 2006, 2005, or some filing system like that. I have thousands of emails, but no one mbox gets any larger than around 2,000 emails. Admittedly, my file structure is very complex.