I noticed that besides the address where people can send bitcoins, there is an even longer number with numbers and letters. So, it's essential for me to save/keep 4 things: 1.) The site or program for the wallet, 2.) the address for receiving bitcoins, 3.) the long letter/number key and 4.) my password. So, the only thing I have to give an outsider is the address for sending me bitcoins.
You got it. You can get as secure as you want. Security can generally be tiered as follows:
High-risk: pooled online wallets
These wallets do not let you extract a private key, so if the site goes down, you lose access to your BTC. Additionally, if the site's compromised and someone can access the admin privkey(s), they can take all your funds, so you're totally trusting in the competence of the admin.Examples: MtGox, most mining pool wallets, MyBitcoin
Medium-high-risk: Individually-keyed online wallets which store privkeys on their server long-term (unencrypted hot wallets also belong here)
These wallets let you extract a private key in case the site goes down, but if the admin accounts are compromised, your BTC can be stolen. A crooked admin can also take your funds at will.Examples: GLBSE v1,
Medium-risk (or medium-low risk depending on implementation): Individually-keyed online wallets (and many lightweight clients) which do not store privkeys long-term (or might use a signature system)
These wallets let you extract a private key and do not have long-term access to your accounts. In some cases they have no access. There may be some dangers of a type of MitM attack either from a third-party attacker or possibly the admin depending on implementation. These usually allow encryption of the wallets.Examples: Strongcoin, Blockchain.info
Low-risk: Encrypted, local hot wallets ("hot" meaning the computer hosting it is connected to the Internet)
This is probably the most popular choice for those willing to host and keep up with the blockchain. You can extract and backup your keys, and if your wallet files are stolen, your funds will probably still be safe so long as your password is reasonably secure.Examples: BitcoinQT
Ultra-low-risk: Encrypted offline wallets
Here, privkeys are kept on a computer not connected to the Internet or your local network. You have an online computer which can broadcast transactions, but these transactions need to be "authorized" by the offline computer. Generally, you use something like a thumb-drive to transfer files back and forth. This still has vulnerabilities, however.Examples: Armory, paper/brain wallets (see Mike's post)
(apologies for any misinfo in advance. I didn't mean to write something so long, and I'm no expert.)
