QColorDialog: fix UB (invalid cast) in Private::setVisible()

The function can be called from ~QDialog(), in which case a cast of
q_ptr to QColorDialog is UB.

Says UBSan:

  qcolordialog.cpp:72:5: runtime error: downcast of address 0x7ffd37909750 which does not point to an object of type 'QColorDialog'
  0x7ffd37909750: note: object is of type 'QDialog'
   fd 7f 00 00  30 34 6e 35 4a 7f 00 00  80 b9 00 00 90 61 00 00  08 36 6e 35 4a 7f 00 00  00 00 f2 a6
                ^~~~~~~~~~~~~~~~~~~~~~~
                vptr for 'QDialog'
    #0 0x7f4a32e8738a in QColorDialogPrivate::q_func() qcolordialog.cpp:72
    #1 0x7f4a32e8738a in QColorDialogPrivate::setVisible(bool) qcolordialog.cpp:2154
    #2 0x7f4a32ea5675 in QDialog::setVisible(bool) qdialog.cpp:757
    #3 0x7f4a306e7768 in QWidget::hide() qwidget.cpp:8179
    #4 0x7f4a32ea4e89 in QDialog::~QDialog() qdialog.cpp:398

Fix by casting to Qcolordialog only on show() (in initWidgets()), not
hide(), and otherwise cast at most to QDialog* (QWidget* would
actually have sufficed).

Add a code comment.

Amends e0bb9e81ab1a9d71f2893844ea82430467422e21 (I think; it might
have been present in a different form before that).

Pick-to: 6.8 6.5
Change-Id: I006406b6cf1012fc3c7a910abcfe14bc119a2b29
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
(cherry picked from commit 0be1ca029cc21e72d9f5408dea99722fba9bc321)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
This commit is contained in:
Marc Mutz 2025-03-22 18:26:46 +01:00 committed by Qt Cherry-pick Bot
parent c831210d0b
commit c50d67e86f

View File

@ -2151,7 +2151,9 @@ void QColorDialog::setVisible(bool visible)
*/
void QColorDialogPrivate::setVisible(bool visible)
{
Q_Q(QColorDialog);
// Don't use Q_Q here! This function is called from ~QDialog,
// so Q_Q calling q_func() invokes undefined behavior (invalid cast in q_func()).
const auto q = static_cast<QDialog *>(q_ptr);
if (visible)
selectedQColor = QColor();
@ -2161,7 +2163,7 @@ void QColorDialogPrivate::setVisible(bool visible)
// Set WA_DontShowOnScreen so that QDialog::setVisible(visible) below
// updates the state correctly, but skips showing the non-native version:
q->setAttribute(Qt::WA_DontShowOnScreen);
} else {
} else if (visible) {
initWidgets();
}
} else {