A faster way to speed up Mail.app
UPDATE: A number of posters in the comments have pointed out that it is possible to run this command on just one line in the Terminal and to leave some of it out. See the comments if you are interested in more fancy ways to get the job done.
As everyone knows, it is possible to get quite a speed boost out of Mail.app by stripping all the bloat out of its Envelope index, an SQLite database Mail uses to store senders, recipients, subjects and so on.
In a past Hawk Wings tip 
, I suggested that quitting Mail, deleting the Envelope file and restarting Mail would force a rebuild that produces a leaner, faster email experience.
In October last year Dallas noticed a faster way to get the same result and posted it in the comments to that tip.
And there it remained until I noticed that Shaun Inman (an iCelebrity and developer of Mint which counts the peeps on Hawk Wings) had noticed it.
Here it is.
1. Quit Mail.
2. Open Terminal.
3. Type the following:
cd ~/Library/Mail
sqlite3 Envelope\ Index
An sqlite> prompt will appear.
At that prompt, type vacuum subjects;.
After a short delay, the prompt will return. Type Control-D to exit.
4. Restart Mail and enjoy the extra speed.
The first time he tried this, Rob Griffiths of macOSXHints reduced his Envelope index from 25.9MB to 4.5MB. My result was less dramatic (21.6MB -> 17.6MB) but Mail.app still felt a lot more zippy.
Some anecdotal evidence suggests that POP users don’t see the same reductions. Can any POP user out there confirm this?
UPDATE: It is easy to automate this using iCal and an applescript. See “Automating the Envelope speed trick“.
Tags: Apple Mail, Apple Mail Tips, envelope, mail.app, POP, Productivity, speed boost, sqliteRelated posts
March 2nd, 2007 at 12:39 am
Hi, I wasn’t clever enough to check the size before I did this, but I do have POP accounts and my Mail.app is zippy now. Thanks for such a useful blog.
March 2nd, 2007 at 1:01 am
I’m using 100% POP3 (only 6) accounts, and this shrank my Envelope Index from 22.3M to 3.4, and mail has never been faster. I did recently purge 20-something accounts and over 8000 messages so my results may be much more drastic than the average user.
Thanks for the tip!
March 2nd, 2007 at 1:06 am
I tried this and - on a POP setup - didn’t see a massive space saving: 27MB before 24MB after.
Things might be a bit snappier now, though. Or I’m hallucinating.
March 2nd, 2007 at 1:16 am
I’m using POP3 with 14 accounts, and my Envelope Index only shrunk about a meg (13.4mb -> 12.2mb).
However, it’s made a very noticeable impact on Mail.app’s speediness - it’s gone from sluggish to snappy!
March 2nd, 2007 at 1:41 am
What exactly is this deleting? I want to try it but only when I know what is happening.
March 2nd, 2007 at 1:44 am
This is the first time i’ve tried using Terminal, so I’m a newbie to say the least. Anyway, I tried to copy/paste the code above into terminal at the prompt, then hit return, i’m getting errors like “SQL error: near “sqlite”: syntax error” and “SQL error: unrecognized token: “\”sqlite>” and when i try hitting control-d, it just beeps. Any help appreciated!
March 2nd, 2007 at 1:58 am
I use 8 pop accounts in mail and I saw an improvement from 1.6MB to 964 KB. it does feel alot faster. thanks!
March 2nd, 2007 at 2:15 am
Ok, guys. Here’s my two cents. And my try.
I did backup it up (I dread the idea of losing my emails…) and gave it a try.
I do have 5 POP account and an Exchange one.
The “Envelope Index” thing was 12.2 Mb before the trick and shrinked as low as 9.7.
And it still - seems - to work
March 2nd, 2007 at 2:22 am
Oh man! Thanks so much!!! So zippier now! (POP)
March 2nd, 2007 at 2:29 am
that is impressive. it made a very noticeable difference.
this is a very good hint.
a big thanks to shaun and hawk wings.
julian
March 2nd, 2007 at 2:31 am
Only shrunk by three megs here, but it is definitely faster when checking my 6+ e-mail accounts.
March 2nd, 2007 at 2:57 am
It does have speed up Mail.app!
Thanks a lot for the trick.
March 2nd, 2007 at 2:59 am
How long does the terminal procedure take?
March 2nd, 2007 at 3:09 am
Took about a minute for me, to go from 24MB to 21MB.
March 2nd, 2007 at 3:16 am
how do u get envelope size ?
March 2nd, 2007 at 3:25 am
Just tested it on my miwed IMAP/POP setup. Envelope Index was 31 MB. Dropped to 25 MB. Speed impact is just incredible. I had pauses of sometimes 2 to 3 seconds when entering the INBOX, now it is just instantaneous.
This maintenance step should be automated on a regular schedule!!
March 2nd, 2007 at 3:32 am
definite improvement on my 3 IMAP accounts and traversing folder structure. thanks for the tip.
March 2nd, 2007 at 3:50 am
Hi, all,
I use POP exclusively (FirstClass is a really lousy IMAP environment…) and saved a whopping 400k by doing this. I went from 14.6 to 14.2 MB. Not really worthy of the effort, I guess.
Then again, after a hard drive meltdown a while back, I’d backed up and reinstalled and reimported all my email only a few weeks ago. It hasn’t had a long time to get full of whatever crud these files get full of.
March 2nd, 2007 at 3:51 am
You can view the envelope size with
ls -lah ~/Library/Mail/Envelope\ Indexin the Terminal - just run it before and after to compare.My (mostly POP) index only went from 16 to 15 MB, but the speed gains are definitely noticable.
I think you can use
sqlite3 ~/Library/Mail/Envelope\ Index vacuum index;to put this into an automated script. (I’d already manually cleaned the database, so I’m not positive this is right. It’s not throwing any errors, and Mail still works right. YMMV.)Great post, Tim!
March 2nd, 2007 at 3:59 am
Took a few seconds (almost all my mail items are local, copied off a POP or IMAP server). 3.6 MB -> 2.9 MB.
But Mail is faster. Awesome.
March 2nd, 2007 at 4:05 am
My Envelope Index went from 28.2 to 7.1 MB. Using 8 POP accounts
March 2nd, 2007 at 4:37 am
Thanks for the tip. I went from 58.4mb to 44.3mb. I have 3 pop and 1 exchange account (Imap).
March 2nd, 2007 at 5:17 am
lucky me i just backed up last night.
7 POP accounts, 1 IMAP. starting size was 10.6MB. finish was 8.9MB. yes, much faster.
also, retrospect tells me the index grew about .3MB last month. fwiw.
March 2nd, 2007 at 5:21 am
Nice tip!
POP from 8mb to 6mb. The application seems faster.
March 2nd, 2007 at 5:29 am
I deleted the Envelope Index manually as I couldn’t get the terminal to work properly.
I use POP3 exclusively and have about 12,000 emails, most of which are in the inbox and sentbox (using mail tags for all my sorting). When I started Mail back up, it reimported all my mail, and everything looked as it should, and indeed, Mail seemed MUCH snappier (and still does), but then I heard airplane sound indicating that mail was sent. As I have now found out, Mail re-sent about 8 messages of varying ages!!!!!!!! It was a little embarrassing in a couple of cases. I was even accused of having a Mac virus!
The other odd thing is that I had a few messages marked as unread!
But I guess it was worth it because mail is much snappier.
My envelope size appears to be much smaller than most are reporting here though. I went from 7.1 to 5.1 megs.
March 2nd, 2007 at 6:02 am
Aaaaa-mazing!
I’ve never touched Terminal before, but my mail.app had slowed down so much that I dived in. Your instructions worked perfectly the first time. I have no idea the size of my Envelope file… I don’t know where it is and I don’t care. But I reopened Mail and the difference is stunning. I recently started doing GTD and as a result deleted about two thousand messages and moved three thousand messages into a new Archives box, and did some other moving around. That’s when my Mail speed deteriorated. I had already re-indexed, but that didn’t help much. I am bookmarking this tip. Thank you Hawk Wings.
March 2nd, 2007 at 6:04 am
Sorry I forgot to say, I use POP mail for three accounts.
March 2nd, 2007 at 6:09 am
Mine went from 90 MB to 83.7 MB but Mail is crazy faster. I have two POP accounts and a .Mac account. Great tip.
March 2nd, 2007 at 6:13 am
“SQL error: database is locked”
How to unlock it?
March 2nd, 2007 at 6:44 am
Went from 3.5 to 2.9 megs, but oh so snappy! Thanks!
March 2nd, 2007 at 7:15 am
Mine has gone from 61.4MB to 60.5 MB.
It took several minutes for the sqlite operation to finish for myself.
However, it has definitely decreased the amount of time it takes to open several of the large mailboxes I have (10000+ messages, mailing lists obviously). That said I delete my Envelope Index file every couple of months because of a bug that makes Mail think that my home directory is full.
March 2nd, 2007 at 7:17 am
Wow.
Background: I just implemented Merlin Mann’s minimalist email procedures for my purely POP set up. Boils down to having only 2 user created folders: archive and act on. Everything else is smart folders. Anyway even though it’s a life changing system (and I do not use those words lightly) I was beginning to feel I was spending a small but noticeable percentage of my time in mail.app waiting for the bigger smart folders to open.
Results: After purging my envelope index every folder (smart or otherwise) opens in under a second.
Yay Shaun. Yay Hawkwings.
March 2nd, 2007 at 7:32 am
On a separate note I get the dreaded “Cannot update your mailboxes because the home directory is full” at least once a day.” Deleting the index file does not do the trick. Has anyone else experienced this problem constantly? Anyone know of a fix?
March 2nd, 2007 at 8:35 am
WOW, I thought I was going to have wait for a new machine to speed up my mail ( I currently have a 933 ghz G4 with a 1.4 acc in it and 1.5GB ram)
My envelope went from 162.3 mb to 137.6 mb
I have 8 accounts -7 POP and 1 IMAP (IMAP is very small though)
This should be regualr maintenace routine for mail.
Thanks
March 2nd, 2007 at 8:53 am
My Envelope File was 66.1 MB. Using “> sqlite3 ~/Library/Mail/Envelope\ Index vacuum index” brought it down to 58.7 MB. (It took quite a few minutes). I then applied
cd ~/Library/Mail
sqlite3 Envelope\ Index
sqlite> vacuum subjects;
This produced no change in the file size at this point.
Frankly, Mail.app is a dog of an app. It causes me more frustration than almost any other Apple app. It seems to have been designed by a MS expat. I hope it is completely rewritten for Leopard.
March 2nd, 2007 at 8:54 am
On my pop machine it went from 84.9 to 71.6, some reduction. Seems a bit faster.
Thanks for the post!
March 2nd, 2007 at 9:09 am
24MB to 13MB and a huge speed increase. Thanks a ton.
March 2nd, 2007 at 9:15 am
Eh, my file was 2,6 mb originally and is now reduced to 2 mb. Thanks though.
March 2nd, 2007 at 10:44 am
I have three POP accounts and a .Mac account. Envelope Index went from 7.1MB to 5.9MB.
No noticable increase in speed.
I was pretty apprehensive trying this tip. The last one (that involved deleting Envelope Index and thereby forcing a rebuild) caused me several hours of grief with lots of files changing status to “… not downloaded from the server…”. It also tripled the number of messages in my system.
Happily this tip has caused no grief.
March 2nd, 2007 at 11:07 am
Wow!! The Envelope Index file only shrunk from 86 MB to 84, but boy is my Mail.app snappy now! Incredible difference! Mail.app has been slow as a pig for me for so long, and now it just flies! Excellent tip!
March 2nd, 2007 at 11:23 am
Yep, works great — amazing speed improvement! I have 7 pop accounts and thousands of archived messages going back to 1997. Everything opens instantly now. Thank you!
March 2nd, 2007 at 11:40 am
It doesn’t work on my system (French OS 10.4.8).
After having typed cd ~/Library/Mail, I get :
[Computer]:~/Library/Mail [Me]$
Then I type :
sqlite3 Envelope\ Index . I have :
SQLite version 3.1.3
Enter “.help” for instructions
sqlite>
And finally :
sqlite> vacuum subjects
The Result :
…>
And nothings happens, it just stay there undefinitely. If I do a ctrl-D, the message is :
…> Incomplete SQL: sqlite> vacuum subjects
If I type :
sqlite> vacuum subjects;
I get :
SQL error: near “sqlite”: syntax error
Is something broken or is it an english-only trick?
March 2nd, 2007 at 11:46 am
Ok, I tried the “sqlite3 ~/Library/Mail/Envelope\ Index vacuum index” command, to see if it worked. It worked flawlessly. I went from a 16.3 MB Envelope to a 14.4 MB Envelope size. Not exactly “earth-shattering”, but what I did notice was the absolute SPEED that Mail.app has now.
I have a 2.0Ghz G5 iMac and here are some statistics for you:
– Before the execution of the Clean-Up code –
*Mail.app would take 3 bounces to load.
*Opening up a folder in it would take about a second to populate the messages.
*It would take about a second to load a message.
– After the execution of the Clean-Up code –
*Mail.app did not even take a full bounce to load.
*Opening a folder is close to instantaneous.
*Opening a message was about a half the time as before.
This would be awesome to have as a script or as a cronjob to happen once a month. A big thank you to the author of this article. THANK YOU!!!
March 2nd, 2007 at 11:51 am
Fantastic tip. Down from 3.6MB to 2.5 with my five POP accounts, and the speed is exceptional. Many thanks for this.
March 2nd, 2007 at 11:55 am
Mail is snappy now, what does it do?
thanks!!
March 2nd, 2007 at 12:00 pm
Brett Peters is correct that it can be done more easily with just a single line…
sqlite3 ~/Library/Mail/Envelope\ Index vacuum subjects;
That’s even easier!
Not knowing too much about sqlite specifically I think the vacuum command just compacts the database, removing uneccessary cruft that builds up through normal use, and potentially reorganizing the data to be more efficient. It’s not the actual amount of disk space regained that’s important. It doesn’t remove any information in the process and it’s a normal part of maintaining a database. Mail really should do it automatically, I think.
The vacuum on subjects is what worked for me before, but a vacuum on index might work for other situations….
sqlite3 ~/Library/Mail/Envelope\ Index vacuum index;
March 2nd, 2007 at 12:05 pm
My Envelope Index was 34M.
I get the following, with no change in the Index size
sqlite3 Envelope\ Index
SQLite version 3.1.3
Enter “.help” for instructions
sqlite> vacuum subjects;
SQL error: constraint failed
March 2nd, 2007 at 12:08 pm
1mb —> 835 kb. looks like i’m a small player
March 2nd, 2007 at 12:13 pm
I’m new to Terminal as well, so could someone please confirm that “…>” means that it is “reorganizing the data.” I was fine up until the sqlite> vacuum subjects part, but now I’m a little concerned that it’s taking so long.
Thank you!
March 2nd, 2007 at 12:23 pm
From the SQLite website:
“VACUUM became a no-op when the GDBM backend was removed from SQLITE in version 2.0.0. VACUUM was reimplemented in version 2.8.1. The index or table name argument is now ignored.”
So it’s probably sufficient to just use “vacuum;” instead of “vacuum subjects;”
March 2nd, 2007 at 12:30 pm
whoa. it only shaved off 5 megs (190.1 to 185), but its a helluva lot faster.
awesome.
March 2nd, 2007 at 12:32 pm
Also worth mentioning is that sqlite also supports an “auto vacuum” mode. I’ve enabled it, but so far I’ve no impression on what impact it has. I can confirm that auto vacuum mode is NOT used by Mail, though, so the vacuum; statement definitely would have an impact.
March 2nd, 2007 at 12:33 pm
My first comment seems to have been eaten: vacuum’s parameter is ignored by sqlite, so there’s no point in vacuuming subjects vs. indexes. Just “vacuum;” by itself will optimize everything.
March 2nd, 2007 at 12:37 pm
According to the SQLite documentation, you don’t need the “subjects” or the “index” at the end of the command.
So it should work with just this:
sqlite3 ~/Library/Mail/Envelope\ Index vacuum;What Vacuum is doing is cleaning up the database. When you delete things, they are simply marked for deletion - not actually deleted. Vacuum does the actual deletion.
According to the documentation, you can set “auto-vacuum” but it has to be done before the tables are created. I bet a script could be made to backup the data, build new tables, set auto-vacuum, and then restore the data. But I am not in need of it that badly. :)
March 2nd, 2007 at 12:40 pm
I haven’t seen the improvements others have, my envelope file dropped a couple of megs from 19Mb to 17Mb but I do use mainly pop3 mail services. Mail seemed to launch faster, but then it always does when you actually time the little blighter.
March 2nd, 2007 at 12:44 pm
Changed from 10 - 9meg. Slight, almost imaginary, change in speed on large smart folders. Real change in search - now usable.
March 2nd, 2007 at 12:57 pm
I did it. My index file went from 13GB to 10GB … and mail was a bit more zippy!
March 2nd, 2007 at 1:21 pm
I need a bit of clarification. Are we talking only about Mail 2? Will this not work on Panther? It worked like a champ on my MacBook. The old G4 with 10.3.9 returns command not found.
March 2nd, 2007 at 1:24 pm
The vacuum command ignores its arguments in SQLite 3, so
sqlite3 ~/Library/Mail/Envelope\ Index vacuum
will suffice.
March 2nd, 2007 at 1:25 pm
Don’t play with terminal at all. When I type in the commands - or when I copy and paste - I get
SQL error: unrecognized token: “\”
Any hints would be much appreciated. I have a big envelope and slow Mail.
This looks like a great hint if I can get it to work,
thanks
March 2nd, 2007 at 1:25 pm
Don’t play with terminal at all. When I type in the commands - or when I copy and paste - I get
SQL error: unrecognized token: “\”
Any hints would be much appreciated. I have a big envelope and slow Mail.
This looks like a great hint if I can get it to work,
thanks
March 2nd, 2007 at 1:44 pm
Wow, thanks for the tip. It really made my Mail.app faster, even though it only went from 5 to 4 MB (2 POP accounts).
This command reclaims space left by deleted elements and reorganizes the data structures. It’s something similar to defrag on file systems. In the latest versions of sqlite, the name of the table (subjects) is not really necessary and is ignored.
I only used simply:
cd ~/Library/Mail
sqlite3 Envelope\ Index
sqlite> vacuum;
Check out the documentation:
http://www.sqlite.org/lang_vacuum.html
March 2nd, 2007 at 1:54 pm
Stuart Luff asked, what the vacuum command is doing. Here is the important paragraph from the sqlite3 documentation:
“When an object (table, index, or trigger) is dropped from the database, it leaves behind empty space. This makes the database file larger than it needs to be, but can speed up inserts. In time inserts and deletes can leave the database file structure fragmented, which slows down disk access to the database contents. The VACUUM command cleans the main database by copying its contents to a temporary database file and reloading the original database file from the copy. This eliminates free pages, aligns table data to be contiguous, and otherwise cleans up the database file structure. It is not possible to perform the same process on an attached database file.”
(source http://www.sqlite.org/lang_vacuum.html)
March 2nd, 2007 at 2:02 pm
What other applications are sql intensive that could benifit from such culling?
March 2nd, 2007 at 2:27 pm
Holy crap! This little trick saved my sanity! ;-)
March 2nd, 2007 at 2:29 pm
Wow - this is great!!!
I’ve never thought of Mail.app as slow — What a difference.
My envelope index went from 12.2mb to 5.2mb.
The 3 line command didn’t do anything for me.
The single line command did the trick.
Thanks guys!!!
March 2nd, 2007 at 2:48 pm
I am running a MBP 2.1ghz Intel (1st version) …
wow.. alot snappier. didn’t look to see if I saved space.. not worried about that with this machine..but wow.. its quick.
March 2nd, 2007 at 2:51 pm
Wow, what a nice tip!
Before: Envelope 14 MB, Mail take 7 bounces to launch, 6 seconds to quit.
After: Envelope 13MB, Mail take 2 bounces to launch, 1 second to quit.
And the search is usable again :)
GREAT!
March 2nd, 2007 at 3:17 pm
Great stuff,
and for those interested in the details, please check: http://www.sqlite.org/lang_vacuum.html
March 2nd, 2007 at 4:16 pm
two IMAP accounts.
Before: 128 Mo
After: 122 Mo.
A small improvement in performance.
March 2nd, 2007 at 4:21 pm
Went from 65MB to 48MB. Dramatic speed increase. Kudo’s to the individuals that figured this out!!
March 2nd, 2007 at 4:31 pm
I’m not sure what the size difference was, but I got a huge performance boost. Thanks much!!
March 2nd, 2007 at 4:35 pm
Seems like this applies specifically to 10.4+. On 10.3 there isn’t the “~/Library/Mail/Envelope” directory nor the “sqlite3″ command, as Daniel noted earlier.
March 2nd, 2007 at 5:07 pm
1 POP account. Envelope Index from 70MB to 69.9MB. Humph. But the non-spotlight searches do seem snappier, and big folders do open a lot faster. Thanks!!
March 2nd, 2007 at 5:51 pm
Oh well… 6.4Mb to 2.9Mb. I have a ton of emails, don’t know why the Envelope Index was so small… Still, nice tip.
March 2nd, 2007 at 6:12 pm
Just a question, is this a “do once” command or should one repeat this once in a while to “clean up” the envelope?
March 2nd, 2007 at 6:52 pm
Awesome, 33MB to 27MB and the speed improvement is huge. Thanks!
March 2nd, 2007 at 6:55 pm
mine went from 58mb to 7mb. i don’t really notice a difference in speed. i have 2 pop3 accounts w/ a crapload of email and nothing else.
March 2nd, 2007 at 7:00 pm
Is this in anyway different than selecting a folder in Mail.app and doing Mailbox > Rebuild?
I understand deleting the envelope file will initiate a rebuild for all folders, but isn’t this whole sql stuff just a tedious way to do something the GUI lets you do with one mouse click?
[BTW, that 'Simon says' thing up there is quite funny.]
March 2nd, 2007 at 7:03 pm
.Mac & another POP account,
173.6M shrink to 173.5M, only 0.1M was cut out, orz….
but the startup speed change of mail.app is noticing!
March 2nd, 2007 at 7:23 pm
Hi !
Nice tip indeed. Since I’m using only POP accounts, I gained only 3 MB out of 14 MB.
I was wondering whether apply “vacuum” command to other tables/indexes would be of any benefit. According to the SQLite documentation, the way this function is implemented in recent versions (2.8.1 and later), the parameter is ignored.
One only has to run “vacuum” without any parameter.
Regards
March 2nd, 2007 at 7:45 pm
I use 4 POP accounts and my Envelope Index went from 42.2 MB to 5.1. BIG, HUGE difference. MUCH faster now. Best tip in a long time.
March 2nd, 2007 at 7:56 pm
Mail slow?
I guess it must have been for a lot of people here, but that’s probably because there’s a lot of disk access involved in the database accesses, rather than being a “dog of an app” as some unkind commenters have called it.
I have a 3.0GHz Mac Pro with a RAID-0 Seagate 7200.10 SATA 3Gb/s dual drive setup. Nothing on this system is slow, EXCEPT when a app asks for the disk while it’s doing something else (even Spotlight indexing). Any app will bog down when drive access is the bottleneck - it’s the weakest point on any modern system.
Ran this DB optimizing command, probably faster, but Mail was pretty fast for me before.
Hope someone forwards it to someone the Map.app development team though!
March 2nd, 2007 at 8:15 pm
Improving Apple Mail performance…
If you regularly use Apple Mail as I do, and you have thousands of mails stored, you will notice that mails may take a few seconds to load, particularly old ones. In my case, I have around four thousand mails, with a envelope database of about 8Mo.
…
March 2nd, 2007 at 8:20 pm
Amazing results! Really!
The whole line all you command prompt newbies out there should use is:
1. “sqlite3 ~/Library/Mail/Envelope\ Index/ vacuum”
You should receive a new prompt stating “sqlite3>” or similar..
2. Ctrl-D to exit the sqlite3 prompt
According to http://www.sqlite.org/lang_vacuum.html , the command “vacuum” no longer takes any parameters so “vacuum index” and “vacuum subjects” should make no difference.
March 2nd, 2007 at 8:35 pm
My envelope index went from 2.1mb to 1.7mb but ever since, my fans have been running faster..cpu 3915rpm, system 2388 rpm and hard drive 2500rpm (normal I think).
How can I go back?
March 2nd, 2007 at 8:49 pm
The search goes fast as light now! (I’m pop user)
March 2nd, 2007 at 9:20 pm
Uaoo! Really faster!! Thanks a lot!
March 2nd, 2007 at 10:04 pm
My fans are ok now after a restart and lots of restoring and importing of mailboxes. Wish I hadn’t started it.
March 2nd, 2007 at 10:16 pm
an explanation of what the ‘vacuum’ command actually does.
http://www.sqlite.org/lang_vacuum.html
March 2nd, 2007 at 10:17 pm
i didn’t check the index size first but it’s much snappier now (11 pop accounts and 1 .mac account), thanks for the tip!
i wonder why there’s no default maintenance script that would be scheduled monthly to speed up mail
March 2nd, 2007 at 10:20 pm
Wow!
I’m running an iMac G4 1GHz (Tiger) and had a Index of 1.9MB, running this took it down to 1.6MB and the speed increase is awesome!
Are there any other apps that would benefit from this!?
March 2nd, 2007 at 10:57 pm
2 POP Accounts, several hundreds thousands email.
Before: 65.2 MB
After: 61MB
But my biggest folder is now fast like Hell!
March 2nd, 2007 at 11:11 pm
This is a killer tip. No placebo effect here Mail.app is much faster displaying mail and switching between mailboxes.
Now I’m looking to see how this could be applied to Yojimbo which also uses an sqlite database and for me at least, is a very slow dog of an app however much I love it.
The Yojimbo database is here:
~/Library/Application Support/Yojimbo/Database.sqlite
Would any sqlite guru like to help me with the syntax? Will it have the same effect?
Cheers, John
March 2nd, 2007 at 11:17 pm
I did not do this command snce my index is only 2.7M and I don’t feel my searching in Mail is slow, but it does make me wonder: I have thousands of archived mail in over a dozen mailboxs from four POP accounts. Why wasn’t my index bigger?
Could there be some other Mac OS X utility that’s cleaning the index?
Michael
March 2nd, 2007 at 11:17 pm
Only 3 POP accounts, 2000 messages. Massive difference in speed. Cheers, Mail had really been annoying me recently.
March 2nd, 2007 at 11:43 pm
no big diff here (on mostly IMAP accounts) 27MB -> 24MB
March 2nd, 2007 at 11:45 pm
For all of you having problems: you’re misunderstanding (not your fault) how instructions are given and how the “type this” parts of sentences interacts with English punctuation…
You can execute this tip in Terminal using just the single line command (copy & paste):
cd ~/Library/Mail; ls -lah E*; sqlite3 Envelope\ Index vacuum; ls -lah E*; cd ~
The above version of the command (1) switches to the Mail directory; (2) displays the Envelope file’s size; (3) reindexes; (4) shows the file’s size again, for comparison; and (5) switches back to your home directory. (Note that my semi-colons in the preceding sentence match the sequence of semi-colons in the unix commands given.)
In other words, when reading the tip, you should be reading very carefully to figure out which punctuation is English punctuation and which is part of the command. For example, when following the instruction in the tip:
type vacuum subjects;.
you should NOT type a period at the end — it’s a poorly placed/confusing English period– but you SHOULD type the semi-colon — it’s part of the command.
To be fair, people who write hints like this and computer tutorials in general really need to take care to distinguish the code from the discussion using new lines and well constructed sentences. .
March 2nd, 2007 at 11:46 pm
Thanks for the tip!
For those of you who wanted to know what the command actually did before running it, here is an explanation from SQLite:
SQLite vacuum command
Remember… you are using SQLite version 3. It does seem unnecessary to use the name of the table because it is ignored in SQLite version 2.8.1 and greater. Oh and make sure that Mail.app is closed before trying this otherwise it might result in an error.
March 3rd, 2007 at 12:29 am
Tim,
You might want to consider a small change to your style sheet for presenting terminal syntax. For example, does the sqlite command include a “.” after it or not? When I ran this command without the period, I got an immediate return to the sqlite command line, so I assumed nothing happened. I re-entered the command with the period, it appears to invoke the command, showing a …> prompt. But there it stays… many, many minutes without giving any indication of completing (an 18.2 MB Envelope Index file). I finally had to quit it to get back to work, and it told me the command was incomplete.
Anyway, your styles are very subtle, and while I can tell the difference between the syntax and your regular body copy in most instances, this is one where I cannot. Perhaps a background highlight color behind syntax?
Cheers!
March 3rd, 2007 at 12:54 am
Two POP3 accounts, ~20,000 emails. Envelope file to start was 18,798,592 bytes, after the vacuum, it was 15,098,880 bytes - a reduction of about 12.5 percent. Not bad.
March 3rd, 2007 at 1:00 am
I do have a suggestion:
Step 0:
Back up your email folder (just copy the whole thing to your desktop) *just in case*
No, my email didn’t get eaten. I’m just paranoid.
March 3rd, 2007 at 1:01 am
Whoever above said this makes a difference to people using IMAP accounts.. I would really take issue with this. My IMAP account hovers around ~250Mb and this doesn’t do a jot for it’s speed. I can appreciate why it works for POP3 however.
March 3rd, 2007 at 1:07 am
Only went from 6.7MB down to 6MB, but the speed difference was still staggering.
I wonder, what the hell was blocking things up?
March 3rd, 2007 at 1:09 am
I saw only a 10% reduction (50 to 45) but noticeable speedup on specific painfully-slow operations. Great tip!
March 3rd, 2007 at 1:11 am
Excellent post - yay! :)
To get around using ^D to exit, try this:
$ sqlite3 ~/Library/Mail/Envelope\ Index vacuum .exit
March 3rd, 2007 at 1:13 am
Wow. Didn’t expect this to work for me - my MacBook Core Duo would open my inbox in about half a second anyway, and my envelope index was a mere 3.4MB.
Now it’s 2.7MB, and after clicking on Inbox it’s absolutely instantaneous. I’m amazed! Great tip.
March 3rd, 2007 at 1:16 am
vacuum subjects; vacuum index;
What these commands do are tell SQLite to re-order the data that’s already in its data store, and in the process reclaim whatever space is then unused.
A bit of background for those who care:
Databases are, by their nature, designed to use a bit more space on-disk than would strictly be required to hold all the data they contain. They do this by making a time-space tradeoff, using more space to reduce the amount of time necessary to use the data they contain. Most algorithms are designed with a balance of space versus time that work for about 95 to 98 percent of uses, and most of the time they work wonderfully.
However, there are times when the data that they put into the databases must be inserted in a space- and time-inefficient manner. This is done, basically, because “you don’t clean house when there’s a customer”… meaning you don’t do potentially time- and memory-consuming tasks when a user is using the application.
In any case, what this command does is take the time- and space-inefficiently-stored user data and rearranges it, by rearranging some of the data which had already been stored as well to make better room for it. Instead of having to basically use an empty storage room that’s not in its normal data structure and use it as a dumping ground for stuff it can’t fit just yet AND have to check that room every time it needs to do something (like ‘load header information to display a mailbox’), it re-sorts what it already has to make room for it in its normal space, and normal catalog.
This makes it take less memory to go through the entire index, makes it have less disk access necessary to go through the entire index, makes it take less processor time to go through the entire index, and makes for a much more pleasantly useful environment.
It is a good thing to do. The thing is that it can’t do it while the user is using the application, because this must be done by a single process which knows where it’s moving everything to and can’t deal with things being put in places that it just cleared to move larger data into.
March 3rd, 2007 at 1:30 am
27->23mb … full pop. No real noticeable speed increase.
March 3rd, 2007 at 1:31 am
It took a little while for the command to go through, but Mail does seem to be quite a bit faster now! FYI, nothing seemed to change inside of Mail appearance-wise, so for other newbies there’s not much worry.
http://www.springboardbusiness.com
March 3rd, 2007 at 1:53 am
Try a 243MB envelope index on for size. Mail is incredibly slow under this load, so I have little sympathy for people with an order of magnitude less data :-)
March 3rd, 2007 at 2:11 am
10.8MB to 10.5MB but did speed responsiveness. Whatever that does, I like it. Seems like something we should do occasionally?
March 3rd, 2007 at 2:24 am
I did the trick with multiple line and the single command “vacuum;”
My Envelope Index trimmed from 90.6 MB to 78.6 MB
Mail.app’s speed is slightly better but I have *tons* of mail :)
Some weeks ago I rebuild spotlight’s index from System Prefs. In *that* case I noticed a great speed bump. Maybe rebuilding those index performs a “vacuum” too.
March 3rd, 2007 at 2:29 am
Anyone know if this has any impact on MailTags metadata? I’ve invested TOO much time tagging messages to lose it all now…
March 3rd, 2007 at 2:36 am
For people who are getting the …> with sqlite, and it doesn’t appear to be doing anything… you MUST put a semicolon at the end of the sqlite command. You know the command is done when you get another sqlite> prompt:
sqlite> vacuum subjects;
sqlite>
If you forget the semicolon, you can just type it at the …> and hit return
sqlite> vacuum subjects
…> ;
sqlite>
and Ctrl-D to exit the sqlite program.
March 3rd, 2007 at 2:41 am
Just a note about what it’s doing…
VACUUM doesn’t delete anything, it merely re-organizes data that already exists.
The size difference of the file itself will depend on how much empty space there was to begin with. A person with a 50 MB index running this might shrink it down to 5MB, a person with a 150MB index might only shrink it down to 145MB.. It’s entirely dependent on the data and how you use your email (people who delete a lot will see a lot more difference, vs. people who keep email around.)
Regardless of the size, speed will be improved - much like any database - when you run an optimization command such as VACUUM. (nicely named, too. Those clever SQLite guys.)
March 3rd, 2007 at 2:46 am
Envelope Index went from 2.0 MB to 1.7 MB, but the speed difference when switching mailboxes is very good. It no longer pauses every when switching boxes to display its contents…it’s instantaneous.
This is on 10.4.8, Mail.app with one POP3 account (gmail)
March 3rd, 2007 at 2:48 am
I’m using Mail.app to perform a daily backup of my Gmail account via POP3 (just one account). I’ve got more than twelve thousand stored mails. My “envelope index” SQL database was 28.9 MB, now it’s 5.5 MB.
March 3rd, 2007 at 3:02 am
Great Tip! I’ve reduced my envelope index from 27MB to 21.7MB. Not really as big a gain as the others but Mail is definitely faster.
I’ve linked this tip from my blog, http://www.theaftermac.com. Thanks Hawk Wings!
March 3rd, 2007 at 3:09 am
This is truly the tip of the day. Mail.app was awfully slow to open my inboxes, in particular the folder that contained all the account-specific inboxes. Now, access is just instant.
Interesting to see that Mail.app uses an SQLite datbase there — this might actually open up some interesting options for automated processing!
March 3rd, 2007 at 3:15 am
I too got an error:
“SQL error: constraint failed”
If anyone knows what to do about this, please post here. Thanks!
March 3rd, 2007 at 3:21 am
Hey guys,
could you expand on this tip. What EXACTLY is happening? And are there any drawbacks? Exactly what data is being lost in the process?
I am a bit wary of apply this to my mail if it effects any of the messages by removing information I may rely on later.
March 3rd, 2007 at 3:29 am
Speed up Mail.app…
Here’s a tip for speeding up Mail.app, in case it appears sluggish. Here’s how I did it on my MacBook Pro:
MaBi:~/Library/Mail ugocei$ ls -l Envelope Index
-rw-r–r– 1 ugocei ugocei 84746240 Mar 2 17:08 Envelope Index
MaBi:~/Library/M…
March 3rd, 2007 at 3:46 am
After reading all of the helpful comments above, I offer a short Applescript to run every now and then. It quits Mail, runs the “vacuum” and shows the file sizes before and after. Copy and paste the following into ScriptEditor:
tell application “Mail” to quit
do shell script ”
cd ~/Library/Mail
ls -lah E*
sqlite3 Envelope\\ Index vacuum
ls -lah E*
”
display dialog result
March 3rd, 2007 at 3:52 am
My current email is at 31,096 messages in the mail app and this helped allot, definitely see a difference in speed. Thanks!
March 3rd, 2007 at 4:04 am
From my console.log, after “vacuuming”:
Mail[pid] Unread count went negative for mailbox JunkHope that doesn’t mean I’m spamming. ;-)
March 3rd, 2007 at 4:20 am
Wow, I was looking forward to this, but apparently my setup is different from everybody else’s?
paul Library/Mail$ sqlite3
bash: sqlite3: command not found
paul Library/Mail$ sudo sqlite3
sudo: sqlite3: command not found
paul Library/Mail$ sqlite
bash: sqlite: command not found
March 3rd, 2007 at 4:22 am
I use POP, and my index dropped from around 2 to 1.4 MB, there was a noticeable reduction in disk “thrashing” during my first mail check of the morning and the download seemed a bit faster. Note that I’m running Mail with SpamSieve on an 2 GHz Intel iMac w/1 GB RAM.
March 3rd, 2007 at 4:35 am
What a great trick! Thanks very much for the tip.
March 3rd, 2007 at 4:58 am
A faster way to speed up Mail.app…
If you’re a Mac user, this little tip can make your bloated Mail.app lean and zippy. Not specifically a blogger tip, but what blogger doesn’t have a bulging inbox?…
March 3rd, 2007 at 5:01 am
Amazing speed improvement.
I have 8 accounts and all with POP, performance improvement was magical.
Thanks for the tip.
March 3rd, 2007 at 5:08 am
http://www.sqlite.org/lang_vacuum.html
All you need is “vacuum;” not “vacuum subjects;”
March 3rd, 2007 at 5:10 am
Now if someone could just do this for Outlook on the PC they’d have the world groveling at their feet. Great find!
March 3rd, 2007 at 5:14 am
I have 6 POP accounts in Mail and a fair amount of messages and the envelope file went from 26.5mb to 3.2mb.
March 3rd, 2007 at 5:42 am
I only have one question. How is it in 2007 people still use POP email? If your messages are important to you the first thing you should do to protect them is switch to IMAP.
March 3rd, 2007 at 6:04 am
The single-line command didn’t work for me (unable to access database) but the multi-line command did work. FWIW.
Envelope went from 9.1mb to 7.7mb, but with dramatically increased speed. Noticeable especially in clearing out SpamSieve spam folder and then deleting trash.
Thanks for the great tip!!
March 3rd, 2007 at 6:24 am
I put together a small Applescript to perform the vacuum. By default it quits Mail, vacuums and restarts Mail. You can customize the script so it just checks if Mail’s running and vacuums only if Mail’s down (for the polite-software lover ;-).
Warning: I’m not an expert! Please let me know if something about this isn’t right. Here’s the script:
– Optimize Mail
– ========
–
– Mar 2007 by Sebastian Morsch
– (based on http://www.hawkwings.net/2007/03/01/a-faster-way-to-speed-up-mailapp/)
–
– Performs the “vacuum subjects” command on Mail’s underlying SQLite database
–
– This must not be done while Mail is running! That’s why the default behaviour of this
– script is to quit Mail, perform the command and start Mail again. Set the property
– “AutoQuitRestartMail” to false if you don’t want the script to do that. It will then just
– detect if Mail is running and won’t do anything if it does.
–
property AutoQuitRestartMail : true
– #############################################
– Main
– Is Mail running?
set mailWasRunning to appIsRunning(”Mail”)
– Quit Mail if we’re supposed to
if mailWasRunning then
if AutoQuitRestartMail then
tell application “Mail” to quit
else
– Mail is running, exit script
return
end if
end if
– Check if Mail was really quit
repeat with i from 1 to 100
if not appIsRunning(”Mail”) then exit repeat
delay 0.1
end repeat
– Timeout: end this script if we waited 10 seconds without Mail quitting
if i is 100 then return
– Do the vacuum
do shell script “sqlite3 ~/Library/Mail/Envelope\\ Index vacuum .exit”
– Restart Mail if we’re supposed to
if mailWasRunning and AutoQuitRestartMail then
tell application “Mail” to run
end if
– Function to determine wether an app is running or not
on appIsRunning(appName)
tell application “System Events” to set IsRunning to (name of processes) contains appName
return IsRunning
end appIsRunning
March 3rd, 2007 at 6:39 am
I get about 100 junk mails a day. I have my Junk Mail folder set to delete email after 1 day old (but it doesn’t actually work). So I’m deleting a lot of email every day.
I noticed a huge speed increase after vacuuming the tables. I wonder why Apple doesn’t do this automatically?
I’m setting up a cron job to do this once a month.
March 3rd, 2007 at 6:50 am
I get the impression from the SQLLite docs that the real problem that the vacuum command fixes is not so much the presence of deleted entries, but rather the fragmentation of the indices.
When mail is deleted, the index entries are marked as unused, then when new mail needs to be indexed, the ld entries are re-used. Unfortunately the index winds up out of order, with pointers going back and forth, just like an MS-DOS file system. You de-frag a file system; vacuum does the same for the SQLList index. It may make things smaller, or may not make much difference in size, but it converts a list which involves hopping mack and forth into something ordered, which is far faster to traverse.
March 3rd, 2007 at 6:58 am
Any reason not to vacuum the other seven tables in that database?
March 3rd, 2007 at 6:59 am
Envelope db size shrank from 11 to 10MB. No difference in opening the application, no difference in search speed and no difference in mail list or message display speed.
All POP3 hitting about 8 accounts with dozens of mail folders.
March 3rd, 2007 at 7:01 am
Definitely speeded up my Mail.app, even with only POP accounts.
So, can you set the SQLite auto_vacuum pragma in this DB and have it do the vacuuming?
March 3rd, 2007 at 7:03 am
For those of you recieving the “SQL error: database is locked” error, simply quit Mail.app and try again.
March 3rd, 2007 at 7:04 am
WOW! Noticeably faster. Thanks!
March 3rd, 2007 at 7:09 am
Sounds like a good Automator Action to me.
March 3rd, 2007 at 7:13 am
I have 2 POP accounts in Mail that have been driving me absolutely batty in its sluggishness (background: I’ve got a 1.66 G4 Powerbook with 1GB). I tried the one-line approach and saved 2 MBs out of the practice (10 -> 8).
Unfortunately, I do not see any difference in speed at all - Mail actually took longer than average to launch (both the first & second time after vacuuming), but opening folders & smart folders and searching take about the same time as before.
Frankly, I’m at wits end with Mail’s lack of speed. Having seen the link on DF to this tip, I totally thought it was kismet as I was planning on googling around later today for faster solutions.
Regrettably, moving to a MBP is not in the budget for a while, but I should rethink bumping the RAM.
Any other tips folks might have would be greatly appreciated in the meantime!
brian AT upsodoun DOT com
March 3rd, 2007 at 7:24 am
I use Mail.app to backup my GMail. My Envelope Index went from 166M to 143M. I have way too much email ;)
March 3rd, 2007 at 7:27 am
Great tip! With 2 IMAP accounts, very noticeable difference. Thanks!
March 3rd, 2007 at 8:08 am
I have about a gig of mail. I’ve been moving folders around quite a bit, and I recently archived and deleted several thousand messages.
Tried out this tip - and Mail’s suddenly a snappy app.
March 3rd, 2007 at 8:18 am
Has anyone measured even wallclock times for doing certain ops that were slow before? Subjective feelings about performance always makes me a little nervous.
March 3rd, 2007 at 9:01 am
Envelope Index 4.9 MB -> 4.1 MB
It’s increbible that a so little tips speeds up my Mail!
Thanks!
March 3rd, 2007 at 9:02 am
Apple should enable the pragma auto_vacuum; unfortunately it has to be enabled before any tables creation.
also, you don’t need to inform the table name. it vacuums the entire database.. from version 3 on the vacuum command doesnt requires any table or index names
March 3rd, 2007 at 9:36 am
I have an IMAP account, and while it made no difference to the speed the account refreshes at, the speed increase in searching is beyond awesome.
March 3rd, 2007 at 9:39 am
Dropped about 1.5MB to 14.5MB (6 POP accounts) and Mail definitely displays folder contents much faster now. Thanks!
March 3rd, 2007 at 9:40 am
@Scott re: MailTags metadata
That’s always one of my first concerns too. No impact in this case. The metadata for MailTags is stored in X-Headers within the individual messages themselves.
March 3rd, 2007 at 9:55 am
My Envelope file is now 3.6MB but that’s for 3 POP accounts with all way over 10.000 e-mails occupying 2GB of space. I don’t know what the size was before but how do you guys actually get Envelope files with over 100MB?!
Anyway, great tip, Mail.app feels a lot faster now!
March 3rd, 2007 at 10:29 am
Wow, this is great. I was just getting ready to go through the pain of clone-off, clone back on my macbook pro hard drive (my previous fix to improving the speed of mail–which only lasts a short time).
Thanks!
Oh–one thing–the article mentioned POP users might not see much improvement, but I am an entirely pop user with many hundreds of thousands of messages and I can report a marked improvement. Actually quite dramatic! Hooray!
Carey
March 3rd, 2007 at 11:31 am
Holy crap! A fabulous speed increase. Many thanks.
March 3rd, 2007 at 12:12 pm
I ran this tonight, but my file was only about 2MB to start with! Almost no change in size, and I use 4 IMAP accounts.
What have you all been doing to your Mail?!
March 3rd, 2007 at 12:31 pm
@Scott Lowe
Just to back up Tim (with a note from the MT developer) This is safe wrt to mailtags — I delibrately avoid hacking internal SQL databases for obvious reasons — MailTags data is stored in X-Headers for IMAP and in emlx files for POP and IMAP (redundancy is good)
While space abhors a vacuum, Mail apparently doesn’t.
Scott Morrison
March 3rd, 2007 at 12:42 pm
Ctrl+D isn’t necessarily the best way to quit software, it may be that sqlite3 considers that signal to be a proper, clean quit request, but I don’t know for sure. Better off to teach unix noobs (most mac users) to use the proper command to quite sqlite3 which is “.q” Yes, that’s a period, then a ‘q’ and the enter key.
- G
March 3rd, 2007 at 12:45 pm
Also, if you REALLY want to speed it up, you can vacuum all indices in the database. Use the command “.tables” to see all the tables. The run “.indices ” for each table. You will see which tables have indices. Run “vacuum ” for those tables.
March 3rd, 2007 at 12:48 pm
Sorry, even better, just type “vacuum;” and it’ll optimize all tables’ indices.
March 3rd, 2007 at 3:34 pm
> I only have one question. How is it in 2007 people still use POP email?
> If your messages are important to you the first thing you should do to
> protect them is switch to IMAP.
Tell that to the thousands of ISPs that provide the email accounts in POP format - there’s bugger all users can do (other than switch to another provider of course).
March 3rd, 2007 at 3:39 pm
A little later to the game, but here’s a better applescript, with feedback.
tell application “Mail” to quit
set sizeBefore to do shell script “ls -lah ~/Library/Mail | grep -E ‘Envelope Index$’ | awk {’print $5′}”
do shell script “/usr/bin/sqlite3 ‘Envelope Index’ vacuum”
set sizeAfter to do shell script “ls -lah ~/Library/Mail | grep -E ‘Envelope Index$’ | awk {’print $5′}”
display dialog (”Mail index before: ” & sizeBefore & return & “Mail index after: ” & sizeAfter & return & return & “Enjoy the new speed!”)
tell application “Mail” to activate
March 3rd, 2007 at 4:30 pm
Ridiculous Internet Handle: I just ran this on my wife’s machine ( 1.25ghz G4 Mini); to open Mail before took four bounces on the Dock, and then another 2-3 seconds before it opened a window. After, one bounce, blam! window open. Now, there may some caching involved there as well, but still, that’s a huge speedup.
What was funny to me was that I was using tab completion to enter the path starting with ~, and when I got to E-tab the whole path changed to /Users/user/… I’d never seen that before. Heh, simple pleasures.
March 3rd, 2007 at 4:50 pm
21.4MB to 7.8MB. 2 active IMAP accounts, 1 .Mac account. 3 inactive IMAP accounts, 2 inactive POP accounts, and a (very large) inactive Exchange account.
Strangely, I haven’t noticed much of a speedup…
March 3rd, 2007 at 5:32 pm
If anyone is interested, I wrote up exactly how to do this and why it does what it does…
http://www.friday.com/bbum/2007/03/02/vacuuming-mails-envelope-index-to-make-mail-faster/
And, no, auto_vacuum could not be enabled in Tiger as that version of SQLite does not support auto_vacuum.
March 3rd, 2007 at 6:09 pm
Thanks!
20MB to 8MB decrease
March 3rd, 2007 at 6:41 pm
Hi,
I deleted around 10.000 mails in the last two weeks and ran vacuum now. 34MB -> 2MB. Maybe these two facts are related?
Cheers.
March 3rd, 2007 at 6:42 pm
Coooool… from 55MB to 50MB but like everyone else, very noticeable speed different. I really hope Apple addresses this in an update to Mail — it obviously really should be routine maintenance!!
March 3rd, 2007 at 7:23 pm
In my case the filesize of Envelope Index didn’t change at all from 23MB, but I never delete anything from my 8 IMAP accounts. Still, Mail.app does seem to be faster now, but that might has something do with me closing and the opening it again (file cache and all).
March 3rd, 2007 at 9:09 pm
THANK YOU This tip sincerely did work and i was contemplating switching to another mail client as mail became so slow and unresponsive at times.
Personally i think that apple makes your mail / safari slower over time and more so before launch of a new os release… you go and get a new OS and bam! everything is fast - when all you really needed was to run a few lines of code to get you back in business :) - just kidding the tip really worked.
March 3rd, 2007 at 11:46 pm
WHOA! One .Mac plus two other IMAP accounts, around 1,375 messages in all (inbox, drafts, sent). Ran the original (vacuum subjects;) and the suggested variation (vacuum;). Modest envelope index size reduction (864 KB to 756 KB), but from clicking the Mail icon to displaying messages is basically instantaneous! (PowerBook G4, 1 GB RAM, 100 GB HDD, 128 MB VRAM; ADSL, Ethernet port). Thanks for the tip!
March 4th, 2007 at 12:35 am
tnx,
I use 5 pop addressen.
Before 4,1 Mb, now 2,2 Mb.
March 4th, 2007 at 12:36 am
How is this different from the Rebuild command under the Mailbox menu?
March 4th, 2007 at 1:52 am
I’ve created a simple Automator workflow for this command. Attaching it to iCal to run once a month or so would be the best use, but it could also be handy saved as an application file. It can be found here: Mail Vacuum
March 4th, 2007 at 4:47 am
How about eliminating the need to run periodic vacuuming? How about not adding an Automator, shell, or AppleScript to the equation? How about…auto-vacuuming? Seems to be working for me.
cd ~/Library/Mail
mv Envelope\ Index Envelope\ Backup
sqlite3 Envelope\ Index “PRAGMA auto_vacuum=1;”
sqlite3 Envelope\ Backup “.dump” | sqlite3 Envelope\ Index
rm Envelope\ Backup
And forget about doing manual maintenance.
March 4th, 2007 at 6:24 am
Used on my POP account. 17.8MB.->7.8MB and much zippier.
March 4th, 2007 at 1:23 pm
The command is not clear. Do I type this in two lines as shown or is it actually one line with no empty spaces?
cd ~/Library/Mail
sqlite3 Envelope\ Index
March 4th, 2007 at 4:33 pm
Does anyone read any of the comments BEFORE they post their comments?
Most of the things have been said over and over, and people still ask the same questions…
Anyway:
bbum said:
Are you sure about that?
According to the SQLite release notes auto_vacuum was added in 3.1 and my OSX 10.4.8 reports Sqlite to be at version 3.1.3:
09:29:10: ~ > sqlite3 -version
3.1.3
Although they do state here that if you are using auto_vacuum that you really should update to 3.1.4 due to a bug…
March 4th, 2007 at 5:16 pm
Doesn’t work. Because the “dump” statement dumps full SQL to create the tables, which then overwrites whatever you did when creating the database - such as setting the auto_vacuum.
So what I am going to try and do is to dump the database to a file, than change the CREATE statements to add in the “PRAGMA auto_vacuum” and then import them back in.
March 4th, 2007 at 6:04 pm
Let me clarify… By doing it that way I could not get the auto_vacuum to stick. No matter what I did, it always reported a value of 0.
I was able to get it to stick by taking these steps, and I am going to simply describe them as I am not sure that people who are not familiar with databases should be doing this until the process is ironed out:
(of course make backup before starting)
1. Open sqlite3 on the “Envelope Index” database.
2. Set “.output” to a file, I called fulldb.sql
3. “.dump”
4. exit sqlite3
5. delete Envelope Index file
6. open sqlite3 on “Envelope Index” and it creates the NEW database.
7. verify empty database by “.tables” - should be none there
8. “PRAGMA auto_vacuum-1;
9. Call “.read” on the fulldb.sql file I created in step 2 and 3.
10. verify data is there by “.tables” or running some SQL statements on the tables.
11. exit sqlite3.
When I do that - now whenever I go back in and:
PRAGMA auto_vacuum;It always shows as 1. Additionally my database is now slightly larger in size as it is supposed to be when “auto_vacuum” is on.
Anyway, that worked for me. I am sure the steps could be documented fairly well, and even put into a script so people could set the auto_vacuum setting themselves.
All of this - just in time for Leopard to come in and change how it all works. :)
March 4th, 2007 at 11:09 pm
50+Mb down to 19!
Amazing difference. Thanks.
mac:~/Library/Mail sharon$ ls -al Envelope\ Index
-rw-r–r– 1 sharon sharon 5050368 Mar 4 23:06 Envelope Index
mac:~/Library/Mail sharon$ sqlite3 ~/Library/Mail/Envelope\ Index vacuum
mac:~/Library/Mail sharon$ ls -al Envelope\ Index
-rw-r–r– 1 sharon sharon 1932288 Mar 4 23:08 Envelope Index
March 5th, 2007 at 12:56 am
Tried it on my Mail which has about 45,000 messages and while it only reduced the size from 44 to 42mb - the difference in speed at opening up folders is unbelievable. I used to click on my sent folder (or any folder) and wait up to a minute for it to finally open and now it takes less than a second - same goes for all my folders.
Finally Mail is running as it should. Let’s hope this is standard in 10.5
March 5th, 2007 at 1:38 am
Slight digression. It would appear that may people like have 10k plus messages in different mail boxes.
I would like to be able to archive old messages and bring them back in for search at will. This way my current messages would be kept to a manageable size.
The dumb method I have is to just quit Mail and move the Archive2006 (for instance) directory outside of Mail’s path; bring it back into Mail’s directory when I need them. This is slow. Any tip?
March 5th, 2007 at 2:34 am
Hack zur Mail.app-Beschleunigung…
Warum nicht zur Abwechslung einfach Mail.app staubsaugen und damit unter Umständen erheblich beschleunigen? Der Trick ist denkbar einfach: 1. Mail.app beenden (und falls nicht sowieso vorhanden ein Backup anlegen) 2. Terminal öffnen 3. sqlite3 ~/Libr…
March 5th, 2007 at 5:42 am
The simple way seems to do same beneficts:
1) In Mail, click the “Inbox” mailbox
2) Menu “Mailbox” > “Rebuild”
3) Wait…
If you do “Get Info” on “Envelope Index” file before 1) and then after 3) you will notice some fat get out of that file.
And, this procedure is suggested as normal maintenance and troubleshooting method for get Mail performance restored to good life.
Apply this for any maibox and folders in Mail.
March 5th, 2007 at 11:55 am
This sped up Mail astonishingly when it comes to clicking on a folder and displaying mail.
The bad news is that my ability to search mail prior to the time I used this is gone. I’m going to try the Rebuild command on the Mailbox menu and see if that helps.
Many thanks,
John
March 5th, 2007 at 2:45 pm
I also am not quite sure what this does (or did) but I took the risk and it seems to have worked! Thanks!
March 5th, 2007 at 5:42 pm
Wow!
I didn’t check the size first, but mail.app is much, much faster now.
Thanks for the tip.
March 5th, 2007 at 9:35 pm
I agree with Alex - blog al Mac. The rebuild method works fine.
March 5th, 2007 at 11:54 pm
Blaas ook eens het stof van je mailbox…
Sinds Spotlight op mijn Mac ook e-mail kan doorzoeken ben ik gezwicht voor Mail.app - Apple’s Mailclient. Een simpele, maar prettige client om mee te werken, die niet helemaal gebouwd lijkt te zijn op mailboxen die ettelijke tienduizenden mails e…
March 6th, 2007 at 9:11 am
In looking at the question: Mail.app -> Rebuild = sqlite3 -> vacuum?
% ls -l ~/Library/Mail/Envelope\ Index
… user admin 27450368 Mar 5 14:46 …
(rebuild my main inbox)
% ls -l ~/Library/Mail/Envelope\ Index
… user admin 27457536 Mar 5 14:48 …
% ls -l ~/Library/Mail/Envelope\ Index
… user admin 27457536 Mar 5 14:55 …
% sqlite3 ~/Library/Mail/Envelope\ Index vacuum
% ls -l ~/Library/Mail/Envelope\ Index
… user admin 27490304 Mar 5 14:56 …
Now that isn’t what I expected!
March 6th, 2007 at 9:35 am
bbum: as of sqlite v3.1 (the version that’s installed on this system, OS X v10.4.8) , the auto_vacuum pragma is supported. the only snag is that the pragma has to be set before any tables are added.
is there a changelog on the Tiger version that specifies that it’s not included? if so, could you provide a link? thanks in advance; I’m not trying to argue, really, just trying to make sure we’re all on the same page and that I’m not blindly doing something that’s doing no good. :-}
March 7th, 2007 at 6:15 am
Wow! I hadn’t realized how slow Mail was until I did this. I have a few POP accounts and the entire application is MUCH faster now. Great tip!
March 7th, 2007 at 7:16 am
This worked really, really well for me. I have very large mailboxes, tens of thousands of messages behind an imaps server and the only two clients that have historically been able to handle this are Mutt (works withough issue) and Mail.app (not as well, but acceptable). After running this little hack my Envelope size went from 144M to 127M, mail was amazingly faster. I have been wanting to learn ruby so I whipped up a little script to automate this, it’s really just an experiment and works from Terminal or X11 now but could easily be modified to work from cron. I may add some backup functionality if I get motivation.
It can be found here.
Please back up before using if you choose to make used of it and be aware that I am not really a programmer so this may not be the best way to do it.
nb
March 7th, 2007 at 7:23 am
Just to correct a detail - sqlite3 on Tiger (3.1.3) does support auto_vacuum and it is turned on for the Envelope Index file (you can test this by running the following command):
sqlite3 ~/Library/Mail/Envelope\ Index “pragma auto_vacuum”
The difference is that autovacuum simply reclaims unused datbase pages and truncates the database file to shrink it as pages become free, a full vacuum actually recreates the entire database file; defragmenting partially used pages and laying out table data across continguous pages in the database file.
March 7th, 2007 at 8:51 am
aswift:
Thanks for the clarifications. So the auto_vacuum will help with size bloat - but the full vacuum will help more with overall performance. Good to know.
We’ll see what the next version of Mail.app brings. :)
And as a side note, apparently this hint also can be used with Aperture.
March 7th, 2007 at 10:42 am
>2) Menu “Mailbox†> “Rebuildâ€
>Apply this for any maibox and folders in Mail.
I would do this if it weren’t always greyed out.
March 8th, 2007 at 6:08 am
I tried the command “vacuum subjects;” at the sqlite prompt. It did speed up Mail but I suspect it also messed up the search function. Meaning I didn’t find all the mails containing a specific word when typing it in the search window.
March 9th, 2007 at 8:25 am
The subject table is not the only one:
.tables shows a list of all of the tables.
sqlite> .tables
addresses messages sqlite_sequence
attachments properties subjects
mailboxes recipients threads
It might be worth to vacuum all of them.
Regards,
Hans Wolters
March 9th, 2007 at 9:11 pm
Hiya,
I have loads of mail delay issues on my mac but I’m using Thunderbird not mail.app.
It’s a bit of a long shot, but does anyone have any idea whether I could do something similar to speed it up..?!
Thanks :)
March 11th, 2007 at 2:25 pm
Off-topic reply, hoping someone might be able to enlighten me about what none of the few search hits did, including what brought me here:
I’m not exactly sure what it means (something to do with X-UID headers being out-of-sync on the server and client?) but it’s been happening intermittently ever since I started using Mail about a year ago. It stopped for awhile, then reappeared soon after switching IMAP servers a few weeks ago.
All along I’ve used fetchmail to retrieve messages from a subset of remote IMAP mailboxes, delivered to a single local INBOX and later processed with a few Mail rules to move some to their final destination mailboxes for local IMAP access. The “went negative” warning always seems to be for those remote mailboxes, which are also active in Mail accounts but not actually read, or those rules has delivered to. I read other mailboxes in those accounts which aren’t retrieved with fetchmail. Since Mail doesn’t support subscriptions it’s forced to seeing the fetchmail-retrieved mailboxes I’d rather it ignore.
That’s the basic gist of it. There are other symptoms and details but this is already long enough. Any info about why that happens and what might be done to stop it would be appreciated… thanks!
Off to read bbum’s article about vacuuming Mail’s Envelope Index to get more informed about what it actually does before deciding whether or not to do it.
March 16th, 2007 at 3:40 am
Jen-
One mailbox at a time. If you highlight all of them, it greys out.
March 18th, 2007 at 12:37 am
I went from 230K to 110K by deleting Envelope Index and opening up Mail.app and letting it import everything, then doing the sqlite and ended up with 110K. Granted, I have about 1000 times smaller database then the rest of you but Mail.app does feel a bit snappier, which I will always take.
March 18th, 2007 at 11:51 am
28Mb -> 24Mb is not a great improvement but I use IMAP for one account and POP3 for two others - I don’t know if that made a difference…
March 19th, 2007 at 11:26 am
Mail has been driving me nuts for a log time with it’s near geological performance. I have just 2 POP accounts and had a 9.5MB file size before that dropped by 50% to 4.3MB after and now that application is flying.
This should be native in the application though, one of the things I hope to see inmproved with Leopard along with Safari’s housekeeping capability.
Dave
March 21st, 2007 at 12:44 am
202.3mb to 177.8mb - and a massive improvement - 10 - 15 seconds in mailbox switches has been reclaimed! It’s fantastic. Now to try the MacBook Pro with the same trick…
(For the record, about 10 IMAP mailboxes, with over 80000 messages)
March 22nd, 2007 at 9:25 am
I want to point out, because some earlier comments might have run afoul of this, and I don’t have time to read all the comments…
sqlite3 is a program. You run sqlite3 (which is the ’sqlite3 Envelope\ Index’ part) and then you issue sql commands to it (which is the ‘vacuum index;’ and/or ‘vacuum subjects;’ part). sql commands end in a semicolon (;).
So if you’ve run sqlite3, and you type in any of these commands, and you end up staring at a prompt (>) as if nothing’s happening, you can just enter a semicolon. Various sql implementations work this way. When they get an incomplete line (that is, one without a semicolon at the end), they assume you want to enter more of the same command. So type a semicolon. :-)
Also, Allan Jenson says this:
> >2) Menu “Mailbox†> “Rebuildâ€
> >Apply this for any maibox and folders in Mail.
> I would do this if it weren’t always greyed out.
The way around this is: You have to do it *per account.* So click the little triangle next to ‘Inbox.’ See your various accounts listed. Click on one, and then Mailbox -> Rebuild won’t be greyed out.
March 28th, 2007 at 11:42 am
How can I reverse this script? Since running this script about a week ago, many people are writing saying that when they reply to emails I send them that their emails are bouncing. I’d rather have slow email and know that I’m getting everything. I definitely did speed up the application, though!
Could this be because I am running both IMAP and POP accounts in Mail?
March 30th, 2007 at 3:43 pm
It is only possible to modify the value of the auto-vacuum flag before any tables have been created in the database. No error message is returned if an attempt to modify the auto-vacuum flag is made after one or more tables have been created.”
possible solution: back up, recreate database, set the flag and copy data from backup. But I won’t try it, too much mail and too little time.
CONVONIX.BLOGSPOT.COM
April 5th, 2007 at 2:45 pm
This command is just as effective to speed up your email and runs much faster than VACUUM:
cat ~/Library/Mail/Envelope\ Index >/dev/null
Like VACUUM, the speed up effect disappears once you reboot your machine.
April 11th, 2007 at 11:19 pm
I tried this and unfortunately it didn’t work. Ever since I installed Google Desktop my mail has been horribly slow.
aL
April 12th, 2007 at 5:58 pm
I used “sqlite3 ~/Library/Mail/Envelope\ Index vacuum subjects;” and it made my envelop size smaller.
But, it seems to have destroyed the index of my Adium chat logs. Adium keeps trying to reindex them, but seems incapable of “saving” its new index, so it just starts over repeatedly. Since they’re not indexed, I can’t search my logs anymore. (No recent Adium upgrades, so I don’t t