On March 28th, 2018 at approximately 1:30 pm CST the server housing the cPanel Feature Request site’s hypervisor unexpectedly failed.
This failure resulted in the loss of data back to September of 2017. Read more in our announcement.
 

Store Email on a different partition.

Chris shared this idea 2 years ago
Open Discussion

IMAP mail has become the norm, with customers needing more and more space for messages. At the same time, SSD drives are now common, which provide a great performance boost for websites and databases but at a lower capacity and a premium cost.


In the current situation, messages are stored in the home partition at /home/username/mail which means that web hosts using cPanel have to store large volumes of mail on the same drives used for websites.


It would make sense to allow the /home partition to run from SSD's but have a separate /mail partition for Email storage that could utilize higher capacity, slower drives.


The proposal is simply to change the location of mail from /home/username/mail to /mail/username - or to specify the mail partition in the basic WHM settings. This would allow the web host to provide more space for Email and to locate it on separate, more appropriate storage. It would also improve performance.

Comments (10)

photo
1

This can be done with hooks and/or mail service configuration changes. For the most part, this is not needed unless the server is severely overloaded, in which case there are other options to consider.

photo
1

I appreciate your opinion and I can see your point of view.


My opinion on your response is that message storage usage and "severely overloaded" servers don't always go hand in hand. Someone wishing to store thousands of messages will not be causing any greater load on the server, or affecting website performance but they simply have a requirement for more space. That space would be better served from a storage array, or a different drive that is separate to the one used for the home directory. There are quite a few people in this feature request system exclaiming that they want to use SSD's to boost performance, but that type of storage isn't really appropriate for storing huge amounts of mail at this point in time, due to low capacity and high cost. I accept that SSD's will eventually increase in size and drop in price, but I maintain that this is still a good idea.


I also agree that it is possible to do this with some modification, but this feature request is specifically asking for that feature to be included as standard. There are lots of things that can be done with customized configurations and scripting, but I think there is an increasing demand to separate out storage for different purposes and this functionality would begin to meet those demands. If you are comfortable making your own modifications via hooks and configuration changes then that's great. I'd rather this functionality was built in.


In a cloud infrastructure, it would be an advantage to store mail on an entirely separate storage pool that is made up of high capacity 3.5" drives and run the home partition from faster SSD storage. On a single server configuration, using a secondary drive or array would serve the same purpose. With improvements in the software used in this industry It is now possible to get more accounts out of the hardware, but with IMAP now being so popular, it is storage space for messages that takes up the bulk of the server's storage in some cases.


I'm of the opinion that if you are running out of space on a server then you may not have balanced that server correctly and there is no point in overselling services so much that you end up with too many accounts and not enough space. The way you design your hardware and infrastructure very much dictates how it gets used. But you can't ignore the fact that, even if a server isn't overloaded, disk I/O from the processing and retrieval of Email can be significant - and while that might not adversely affect performance, if that I/O was removed from the main /home partition it would provide a performance boost for websites and allow for more accounts to be run from the server, which is nothing to sniff at. At the same time, you can't ignore the fact that in many cases, message storage requirements now outweigh all other capacity needs. It's about getting the right balance, running the right number of accounts on your servers and different types of clients often from different hardware - and I strongly believe that implementing a change like this would go a very long way to making that balance much easier to achieve.


Personally, I'd like to offer my clients super fast, low latency storage for their websites and databases but run their message storage from higher capacity, more affordable drives. I don't think I'm alone in that requirement but I'm also very open to any opinion anyone would like to share.

photo
1

"not needed unless the server is severely overloaded" => not true, as many said we would like to store websites on a fast SSD disks and email on standard hard drives.

photo
photo
2

I would also like to see the ability to store emails, especially the archives on a separate partition. This way, expensive SSD's can be used to service content (Web / SQL / IMAP / etc), while cheaper SATA HDD's can be used as storage.


Usting hooks todo this will probably break things like backups, moving accounts to another server, etc.

photo
2

Yes, that is correct. I've seen a number of hosting providers doing this and is impossible to migrate an entire account: the email accounts are not migrated and the restore process throw errors in regards of non-existent mail folders. Also, backups will not include email accounts contents.

photo
photo
1

I think the larger benefit to something like this is actually preventing something like a wordpress exploit from allowing hackers to siphon a user's email. In addition to SoftDux's note re: storing mail on larger partitions.


Either way, I think this, coupled with some sort of native email clustering would be extremely pertinent.

photo
1

This is a great idea and just what we were looking for. With products like this in the market ( [link omitted] ) we would be able to utilse the SSD for web and HDD for email.

Moderator note: Removed link to external commercial site

photo
1

As soon as this is implemented i will be migrating to cPanel/WHM for all services.

In my experience, 90% of clients cPanel space is dedicated to emails.

This suggestion together with mailbox compression (this feature is in pre-release) would improve storage flexibility, scalability and performance when using HDD for emails and SSD for everything else.

photo
1

This is similar to what I posted here:

https://features.cpanel.net/topic/set-different-mail-storage-path-for-specific-plans


In my case I don't mind hosting email for all users on the regular storage for most users, but there should be an option to use another partition like you said for different plans or accounts (business email). This way, you could have a cPanel hosting plan with bigger email accounts, and normal users upgrade to those plans and now they are using the cheaper storage.


I absolutely 100% agree with you. You can't offer decent email accounts from cPanel today and I suspect cPanel is losing customers based on this. Alot of customers are just migrating to Gmail or Office 365 because they offer huge sizes for email account. Email storage is budget storage, not high performance SSD on RAID setups.


Or at least another solution for now, if this is way too complicated for cPanel to implement, is archiving older emails to another partition. That way, regular emails that need frequent access are still on the regular storage, but very old emails that usually are not accessed frequently go to a cheaper storage automatically.

photo
3

While I like this request a lot ..chances are it will never get implemented. However there is a silver lining ... this is coming in the 11.66 release due in a couple of weeks https://features.cpanel.net/topic/add-option-to-compress-mdbox-in-v58


Basically native Dovecot mail compression which should save up to 90% of the disk space occupied by email storage.