Android is a multi-tasking operating system, it will leave applications running in background even if you don't use them. This is usually good because you might want to return to a running application, or you may want it to "work" while in background (i.e. listening to radio). Sometimes though you might want to force closing an application, to do that:
- Press the Menu key when you're at the Home screen, select "Settings" then "Applications" and then "Manage applications"
- Select the "Running" tab on top
- Locate the application that you would like to close, select it, and then click the "Force stop" button
you can find more details about multi-tasking in Android.