Refactoring is the work that never makes it onto the sprint because it produces no user-visible output. It is easy to defer indefinitely. The cost of that deferral is slow feature development: engineers spend more time navigating confusing code, more time debugging because the structure obscures what is happening, and more time in PR review because reviewers cannot tell if a change is correct.
When refactoring does happen, it often creates large diffs that are hard to review safely. A single refactoring PR might touch 30 files. The reviewer cannot hold all of it in their head at once, and the chance of approving something that breaks a non-obvious dependency is high.