The WordPress media uploader allows you to upload any type of media: images, videos and audio.

WordPress Media Uploader
The uploader is very easy to use, you can either drag and drop files into the designated area, or you can select a file using the regular method.
One problem you might face right away is that the uploader has a size limitation, this can be as low as 2mb. This limit is not easy to change and you will likely have to contact your host to make the change on your behalf.
Another problem with the uploader is that while it will allow you to upload video and audio, WordPress doesn’t have the ability to embed the media (except for images). For this reason I recommend if you do want to upload video or audio I would recommend you use a different service such as YouTube or Vimeo and then embed the media into your post. This is not only easier, but it also uses their bandwidth instead of yours.
If you don’t want to upload your media to a site like YouTube you may be interested in VideoPress and Audio Player Plugin.