For average individuals there isn't much point to this apart from the small interest.
For large amounts of money, there is the problem that bank insurance only applies up to a certain dollar value - currently $250k. Beyond that you're exposed to the risk of the bank defaulting.
Keeping large amounts of physical cash is even more of a risk, against threats internal and external. And it costs money (staff and security time) to move it in and out. Cash handling is expensive even in businesses that do it all the time like supermarkets and casinos.
If you have a lot of money, or if you are the bank, then bonds are much simpler and less risky. They're also available in conveniently large denominations.