I second getting rid of the rudder. Pain in the ass will get broke off eventually anyway. They don't really help much either. Take a look at some of the photos here and you'll hardly ever see one. Literally hundreds of boats and very few with rudders.
Check your grate/loader too. Make sure it's not broken or cracked at the welds. This will definitely cause you problems.
The rudder depends on the boat. I had a CP that when they broke off you couldn't tell (used 2). My Carrera bow rider on the other hand with out it gets squirelly. I have a ride plate, Droop, and diverter on the Carrera.
With out the rudder it acts almost the same as to much diverter, without the porposing.
I am not sure if what you are describing is chine walking, but you might be able to improve it if you can get the bow down.