Openssl is a library that is shipped with a lot of OSes and basically allowed an attacker to dump 64Kb of your memory and it could do it in a loop this would allow access to entropy and store variable like SSL private keys. This is really an issue of putting too much trust in a single library.
That's not quite technically accurate. The attacker cannot access your memory, it accesses the memory of the web server the attacker is connecting to. It can only grab what is in memory. Although the 64K of RAM it grabs is completely random, there is no limit to the number of times this random data could be requested and no way of knowing which data the server is giving out. It is possible that there could be a password or digital key in that random chunk of RAM (and keys are usually pretty easy to find because they fit a specific pattern). This would put you at risk because the attacker would now be able to access that site as you or he could use that key to put himself in the middle of you and the webserver and record everything you do from that point on. Changing your password on that site would only fix the problem if the administrators of that site have fixed the vulnerability. If they have not, then changing your password now could put you at even greater risk.
By now, most large institutions have fixed this issue. If you use LastPass, they have implemented a check to let you know if websites they store passwords for are still vulnerable. There are also other websites out there keeping track. It is a good idea, if you're very concerned about a particular website, to verify that they have fixed their servers before logging in yourself and putting yourself at more risk.
If you have not accessed a certain website in a very long time, then it is unlikely that you would be at risk on that website. However, this vulnerability has existed for up to two years before being publically discovered this week. So it is possible that some hacker out there has known about it for some time and has been exploiting it for many months.
There's a pretty good and simple illustration of how Heartbleed works here:
http://xkcd.com/1354/
