Because of the DAG file size (over 4GB) you can't mine Ethereum (ETH) with only 3GB VRAM. That's the reason why you got the message (out of memory). If you would have a 1060 modell with 6GB, it would be no problem.

The DAG file of Ethereum Classic (ETC) is below 3GB, but I have no knowledge whether it is profitable to mine.

Your graphics card does not have enough VRAM. You cannot use it for mining.