The little one sounds like s/he's being bullied.
If you calculate given the above guideline, and were to try to keep all 3 in the same tank, ideally about an 80-gallon tank/container. I'm not sure 80-gallon tanks are the norm, so go larger, a 90-gallon tank.
Some people calculate the space for multiple turtles a bit differently, allowing half as much the space for a additional ones (but there's a limit to how far this can be done). More space is better, and it is possible that with even more space the bullying will continue. Providing places in a larger tank for a turtle to hide/get away from the others is good, as is providing objects that will help break of their sight lines (so they don't have to look at each other all the time).
There's also a rather large size difference between a 1 and 4-inch turtle. You might want to separate the smallest one from the other two and give him his own set-up until he's bigger. The other two, if kept together still need a larger tank/container (the 20-gallon tank is adequate for a turtle with a shell length of 2 inches).
