Your final proposal (put an observations "on the edge" of intervals into both intervals) is unconventional, but probably worth trying. Note that ordinal logistic regression need more parameters than linear regression, so be careful adding too many categories because your data is small.
You can consider transforming the prediction target to reduce the distance between the extreme high values and the rest of the data.
In general, however, you will need to understand why those extreme high values are high. Are they incorrect entries (i.e. outliers)? Or is there some feature in the data that can be added to the model in order to better predict when a data point is higher than expected?
Let's say you are trying to predict student performance on a linear algebra midterm exam. Your data set consists of last year's exam scores. Maybe your model includes these features:
- Student class year (freshman, sophomore, junior, senior)
- Did student take honors calculus? (binary)
- Average homework grade
- Recitation attendance percent
Your model generally does well. Most students get in the 75-90 range, but some do unexpectedly well and score in the high 90s, almost perfect. Your model fails to predict these students' grades. So what went wrong? One possibility is that there is some other attribute shared by the high-scoring students that the other students might not have. You look back over the exams and realize that all the high-scoring students did well on a tough problem that you had covered in an office hour session. Now you add a new feature to the model:
- Attended my office hours session on October 2nd.
and now your model correctly predicts the high-scoring outliers.