Let's go over the process:
If you have 8 products in a category and you are showing 4 products per page, your product category page will show a blank space like this:

Of course, every store owner would like to setup pages of the store site in such a way that complete site looks filled up without any blank spaces. In order to overcome with this, we can setup each category according to number of products.
Go to: Administration > Catalog > Categories > List View

Choose the category for which you would like to setup the page size.

Now, scroll down as the bottom and you will find this option:
Page Size options (comma separated): Comma separated list of page size options (e.g. 10, 5, 15, 20). First option is the default page size if none are selected.
Change the page size options as per your requirements like this:

The output should be like this:

A copy of this article is also available on Arvixe Blog.