Firstly, the kind of savings group that gives out loans isn't the one you described, rather it's the group that every member contribute certain amounts weekly/monthly and keep it for financial dealings of which everyone gets their capital, as well as profits(with respect to amounts contributed) from financial activities at the end of the savings period.
If that's the case, then savings groups wouldn't give members huge loans unless they've valid collaterals, but they can manage soft loans based on your credit record. Another advantage is that it's collateral evaluation process cum funds distribution is faster compared to bank loans.
I would guess that when it comes to participating, there could be rules. It means that if the people could have multiple key holders, as in money can't be taken out quickly, then we could have a group where it has to be a vote that requires any money going in or out. This way you can prevent unwanted situations like those.
I am not saying this would be the ideal situation, but it would not be a terrible one either; it could work very well if it's done good. This isn't a simple task, but it isn't a very difficult one neither, you can arrange the system that way.