Two reasons (at least).
One is that long threads can overwhelm browsers on small mobile devices, as well as on slow cell or net connections. You may just want to read the two posts that got added in the last 24 hours, but the browser still needs to download everything.
Another is that it's cumbersome to read if you're doing anything besides reading the last few posts on a thread. This becomes a tradeoff between knowing that the particular answer you want is 25% of the way down the page versus knowing that it's on page 2 of a 4 page thread. Particularly if the thread grows, as once you know it's on page 2, it will always be there, but if it's all on one page and the thread doubles in length, it will only be 12.5% of the way down the page. (This feels vaguely familiar; forgive me if I've made this argument previously here.)
I have no idea how many people read and navigate the way I do, versus using forums in ways that might favor single pages. But I've been on the net since the days of the ARPAnet, and the only argument I've ever found in favor of putting it all on one page is that too many net newbies don't understand the concept of having to manually move to the next page of a thread. Then again, I stopped following such usability debates about ten or so years ago.