I believe in current code the upper bound on the fee with default settings is 0.05 BTC— above that the transaction would have to be bigger then the client is willing to generate. (it would take a great number of ins/outs to hit that but I'm not sure how many— but since it depends on inputs too it's hard to give you a simple function)
Thanks for your answer, gmaxwell. I've read about 0.05BTC limitation on wiki faq, but I am not sure if this info is reliable and up to date.
Anyway, this logic seems mysterious to me. Even though Gavin said there should not be any limits, I assume there is some implicit limitation for amount of outputs (which could come from limitation on block size, or transaction size, or fee, or willingness of bitcoin software to generate the transaction).
I am curious if combining many send transactions into a single sendmany is always a preferable way of sending out money to many recipients, or if there is some point at which we should split the transaction into several sendmany transactions. If there is such a point, then how many recipients should be in each transaction?