Could someone explain how to use BIP-37 to get the balance at an address?

[hhanh00]

I did the following:
1. Send a version msg with relay = 0 (don't get inv messages until filterload)
2. Response to version from node with verack
3. Wait for verack from node

Now I'm ready to talk to the server
1. Send an empty filterload
--> I start getting some Inv messages back. I thought I shouldn't get any but maybe it's for DoS protection?
2. Send a filteradd message passing a pubkey where I've got a few mbtc

045e2a21f9e0de43b9ff9c898efcaffd45c78a3b12161fe834266fc9f34c186e6b7de09942beb8e d7039a61d146330ca286f2fb86fec6a181f96d9c59d13695e6d

Still getting inv messages...
Basically, how should I get the actual transactions so that I can calculate the balance?

Thanks,
--h