After a while, you will notice an exponential growth of used space, without even copying anything to it. How so?
Your server size grows mostly because:
- You are adding/updating new packages – cached package data is stored
- Kernels are updating regularly – old ones are kept
- Log files are being created and archived for a long long time
Keeping the packages clean
I’m using this command to clear the apt-get cache clean:
sudo apt-get autoremove & sudo apt-get autoclean & sudo apt-get clean
You can automate this task by adding this entry to the crontab. (sudo crontab -e):
0 11 * * * sudo apt-get autoremove & sudo apt-get autoclean & sudo apt-get clean > /dev/null 2>&1
This will run the command every day at 11AM. I choose this time, as I have other cron-jobs running at different hours.
Remove unused kernel files
WARNING: Please be very careful when deleting kernels. Always make a back-up of your droplet before running these commands, just in case.
After some research I have found this really neat command by David Kemp on askubuntu.com:
echo $(dpkg --list | grep linux-image | awk '{ print $2 }' | sort -V | sed -n '/'uname -r
'/q;p') $(dpkg --list | grep linux-headers | awk '{ print $2 }' | sort -V | sed -n '/'"$(uname -r | sed "s/\([0-9.-]*\)-\([^0-9]\+\)/\1/")"'/q;p') | xargs sudo apt-get -y purge
Cron version:0 10 * * * echo $(dpkg --list | grep linux-image | awk '{ print $2 }' | sort -V | sed -n '/'uname -r
'/q;p') $(dpkg --list | grep linux-headers | awk '{ print $2 }' | sort -V | sed -n '/'"$(uname -r | sed "s/\([0-9.-]*\)-\([^0-9]\+\)/\1/")"'/q;p') | xargs sudo apt-get -y purge > /dev/null 2>&1
Logs and log rotation
In ubuntu there is a built in service called logrotate, which is configured to rotate the log files for other services. It’s configured for each service individually. To get to the logrotate configuration directory run:
cd /etc/logrotate.d/If you now run an ls command, there will be a few files for each important service. We are going to check the apache2 config file, as it’s the most important, and it can grow really quickly to a tremendous size – however in most cases it’s not needed.
Let’s open the apache2 config file shall we:
sudo nano /etc/logrotate.d/apache2After opening, you should see something like this:
/var/log/apache2/*.log { weekly missingok rotate 52 compress delaycompress notifempty create 640 root adm sharedscripts postrotate if /etc/init.d/apache2 status > /dev/null ; then \ /etc/init.d/apache2 reload > /dev/null; \ fi; endscript prerotate if [ -d /etc/logrotate.d/httpd-prerotate ]; then \ run-parts /etc/logrotate.d/httpd-prerotate; \ fi; \ endscript }Okay. So the important parts for us are the following lines:
- weekly – meaning the log files are rotated every week
- rotate 52 – meaning that 52 rotations are kept, thus 52x1week = almost a year
For me this is too much information to keep for no reason. My final configuration is:
/var/log/apache2/*.log { daily missingok rotate 14 size 1M compress delaycompress notifempty create 640 root adm sharedscripts postrotate if /etc/init.d/apache2 status > /dev/null ; then \ /etc/init.d/apache2 reload > /dev/null; \ fi; endscript prerotate if [ -d /etc/logrotate.d/httpd-prerotate ]; then \ run-parts /etc/logrotate.d/httpd-prerotate; \ fi; \ endscript }
There is one more parameter I appended after the rotate line, which is size 1M. This will make sure the rotation occurs once the current log file exceeds 1MB of data. Keep in mind, that the original configuration would have rotated only if the log exceeds 100M by default. This could have potentially created over 5+GB of overhead from log files. Instead, now it caps at 14x1MB.
Comments 10
Wohh just what I was searching for, thank you for posting.
Everything is very open with a really clear description of
the challenges. It was really informative. Your site is useful.
Thanks for sharing!
thanks
Haha, thanks! I am at Siteground although but did not know that the “basic” Ubuntu commands are working on web as well (why not as both linux/unix)
I am checking now your search plugin (lite version at this moment) for my site Ernest. May I will have question later if I stuck with.
Interesting info. Thanks for share
Thank you, Ernest.
Could you please write a tutorial about this guide on Centos?
I’m running a VPS on Vultr, lowest price, store alot of images. Images storage is about 10GB, but it alway notice that my VPS is running out of storage.
I think centOS no need 10GB to run, so may be cached files.
I’m searching but no luck/
Author
Unfortunately I have no experienve with Centos. Try checking server log files for clues. Maybe the log files archive themselves taking a lot of space.
thank you for posting!
Interesting info!
Nice