It's possible that your Cat5 extender is not allowing enough signal to pass through. You may want to consider an active (powered) RS232 extension system from manufacturers such as Neteon or Moxa. They require power on both ends but are more robust and reliable than a simple passive adapter.