Old battery's probably swelled up and that's why it's getting stuck. Don't be shy about removing the entire top cover and front of the frame, if needed.

For mission critical business systems, UPSes are a critical necessity as a single brown out, power surge or power outage could easily cost thousands of dollars worth of lost time, lost data and ...