it's call pagination.

short version:
you need to get the total number of entries (pages) in the database, how many you want to list at a time, the starting and ending points to your list (if you list 25 per page, page 3 starts at entry 75 and ends at 100).

setting up the pagination links is easy, that's just some basic arithmetic. integrating all the logic into the rest of the pages to be handle the dynamic content is another can of worms.

it requires a database, common web usages is mysql and a server side language like ASP (vbscript of c#/.net), php, java, etc.