Being cautious is correct. Being unpredictable because you’re driving abnormally is super dangerous.
The reason we have a driving system is so that everyone knows that to expect from everyone else. If you operate outside of that system you’re a danger to yourself and everyone else.
It sounds like you’re disabled in this area, and it’s unfair that the world is not more accommodating.
“Change the world to accommodate my disability and I’ll quit being a hazard to people’s life, limb, and property” is not an acceptable attitude.