Fix potential 16-bit integer overflow

When multiplying a float in [0;1[ with (1<<16), with rounding, it might
end up being rounded to 65536 even if the input was under 1. This patch
uses a floor operation to make sure the value can be in a ushort, and
cleans up the surrounding code so it is clearer what it does.

Task-number: QTBUG-68360
Change-Id: I2d566586765db3d68e8e7e5fb2fd1df20dabd922
Reviewed-by: Eirik Aavitsland <eirik.aavitsland@qt.io>
This commit is contained in:
Allan Sandfeld Jensen 2018-05-22 16:57:13 +02:00
parent c416a7f257
commit b9bc6c31a0

View File

@ -3193,13 +3193,13 @@ static const QRgba64 *QT_FASTCALL fetchTransformedBilinear64(QRgba64 *buffer, co
const qreal px = fx * iw - qreal(0.5);
const qreal py = fy * iw - qreal(0.5);
int x1 = int(px) - (px < 0);
int x1 = qFloor(px);
int x2;
int y1 = int(py) - (py < 0);
int y1 = qFloor(py);
int y2;
distxs[i] = int((px - x1) * (1<<16));
distys[i] = int((py - y1) * (1<<16));
distxs[i] = qFloor((px - x1) * (1<<16));
distys[i] = qFloor((py - y1) * (1<<16));
fetchTransformedBilinear_pixelBounds<blendType>(image.width, image.x1, image.x2 - 1, x1, x2);
fetchTransformedBilinear_pixelBounds<blendType>(image.height, image.y1, image.y2 - 1, y1, y2);