Saving space automatically – Beginners guide to Digital Ocean

Ernest Marcinko Hosting, Tutorials, Wordpress 10 Comments

After a while, you will notice an exponential growth of used space, without even copying anything to it. How so?

Your server size grows mostly because:

  • You are adding/updating new packages – cached package data is stored
  • Kernels are updating regularly – old ones are kept
  • Log files are being created and archived for a long long time
Keeping the packages clean

I’m using this command to clear the apt-get cache clean:

sudo apt-get autoremove & sudo apt-get autoclean & sudo apt-get clean

You can automate this task by adding this entry to the crontab. (sudo crontab -e):

0 11 * * * sudo apt-get autoremove & sudo apt-get autoclean & sudo apt-get clean > /dev/null 2>&1

This will run the command every day at 11AM. I choose this time, as I have other cron-jobs running at different hours.

Remove unused kernel files

WARNING: Please be very careful when deleting kernels. Always make a back-up of your droplet before running these commands, just in case.

After some research I have found this really neat command by David Kemp on askubuntu.com:

echo $(dpkg --list | grep linux-image | awk '{ print $2 }' | sort -V | sed -n '/'
uname -r

'/q;p') $(dpkg --list | grep linux-headers | awk '{ print $2 }' | sort -V | sed -n '/'"$(uname -r | sed "s/\([0-9.-]*\)-\([^0-9]\+\)/\1/")"'/q;p') | xargs sudo apt-get -y purge
Cron version:

0 10 * * * echo $(dpkg --list | grep linux-image | awk '{ print $2 }' | sort -V | sed -n '/'
uname -r

'/q;p') $(dpkg --list | grep linux-headers | awk '{ print $2 }' | sort -V | sed -n '/'"$(uname -r | sed "s/\([0-9.-]*\)-\([^0-9]\+\)/\1/")"'/q;p') | xargs sudo apt-get -y purge > /dev/null 2>&1

Logs and log rotation

In ubuntu there is a built in service called logrotate, which is configured to rotate the log files for other services. It’s configured for each service individually. To get to the logrotate configuration directory run:

cd /etc/logrotate.d/

If you now run an ls command, there will be a few files for each important service. We are going to check the apache2 config file, as it’s the most important, and it can grow really quickly to a tremendous size – however in most cases it’s not needed.

Let’s open the apache2 config file shall we:

sudo nano /etc/logrotate.d/apache2

After opening, you should see something like this:

/var/log/apache2/*.log {
        weekly
        missingok
        rotate 52
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
                if /etc/init.d/apache2 status > /dev/null ; then \
                    /etc/init.d/apache2 reload > /dev/null; \
                fi;
        endscript
        prerotate
                if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
                        run-parts /etc/logrotate.d/httpd-prerotate; \
                fi; \
        endscript
}

Okay. So the important parts for us are the following lines:

  • weekly – meaning the log files are rotated every week
  • rotate 52 – meaning that 52 rotations are kept, thus 52x1week = almost a year

For me this is too much information to keep for no reason. My final configuration is:

/var/log/apache2/*.log {
        daily
        missingok
        rotate 14
        size 1M
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
                if /etc/init.d/apache2 status > /dev/null ; then \
                    /etc/init.d/apache2 reload > /dev/null; \
                fi;
        endscript
        prerotate
                if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
                        run-parts /etc/logrotate.d/httpd-prerotate; \
                fi; \
        endscript
}

There is one more parameter I appended after the rotate line, which is size 1M. This will make sure the rotation occurs once the current log file exceeds 1MB of data. Keep in mind, that the original configuration would have rotated only if the log exceeds 100M by default. This could have potentially created over 5+GB of overhead from log files. Instead, now it caps at 14x1MB.

Chapters

Comments 10

  1. klik365

    Everything is very open with a really clear description of
    the challenges. It was really informative. Your site is useful.
    Thanks for sharing!

  2. kemeng

    Haha, thanks! I am at Siteground although but did not know that the “basic” Ubuntu commands are working on web as well (why not as both linux/unix)

    I am checking now your search plugin (lite version at this moment) for my site Ernest. May I will have question later if I stuck with.

  3. Melani Perez
  4. Huy Hoa

    Thank you, Ernest.
    Could you please write a tutorial about this guide on Centos?
    I’m running a VPS on Vultr, lowest price, store alot of images. Images storage is about 10GB, but it alway notice that my VPS is running out of storage.
    I think centOS no need 10GB to run, so may be cached files.
    I’m searching but no luck/

    1. Ernest Marcinko Post
      Author
  5. MaddisonWells94
  6. MaddisonWells94

Leave a Reply

Your email address will not be published.