If your WordPress website is going to offer content in a blog or news format you will need to use posts, if your website will mostly offer static content (ie a company website), then you will primarily use pages.
When you publish a post it is displayed on your homepage and your RSS feed in reverse chronological order, this means the newest content is displayed at the top.
Unlike pages, posts can be organised by categories and tags.
Categories and tags can be useful if you have a lot of content and need a way to organize it. For example this chapter has a category of it’s own: