Set up a line drawing on a curve with an unknown number of points

I have a sample curve that theoretically ends up with decreasing exponent. The end of the curve gets caught in the noise. Examples of points are shown in the journal scale. What I want to do is find and fit the linear part of the log curve to get an exponential factor. The trick is that I don't know the start point or end point of the linear portion of the logarithmic curve.

The strategy I use is to place a line at each point at least 20 points until the end of the curve is reached. Then of all these regressions, I keep the one with the best coefficient of determination.

I did some tests and found that the RMS error increases with the number of points systematically as the curve gets more noisy, so the extracted coefficient is always calculated on the minimum number of samples (20 in this example).

My question is, is there a more efficient method for calculating this factor? Does the minimum number of samples increase the accuracy of the fit?

Here's a sample curve I want to set

Calculated slope relative to position on the curve

And the corresponding RMS error