This amends commit 394788c68efacdec2676988b4b4ff207b20557f2 (its
ChangeLog applies to this commit too). That fixed sorting of UTF8-to-
UTF16, but when adding more unit tests, I've discovered that some UTF-16
strings also sorted incorrectly. There were two problems:
First, we were assuming that we could rely on the UTF-16 length as a
proxy for the UTF-8 one, but that's not true for some cases:
* both 1-, 2- and 3-codepoint UTF-8 sequences are 1 codepoint
in UTF-16, so some strings would have identical UTF-16 length
* 4-codepoint UTF-8 sequences shrink to 2-codepoint UTF-16 ones
(2:1) but 3-codepoint UTF-8 sequences shrink to 1 (3:1), so
some strings would be longer in UTF-16 but shorter in UTF-8.
Second, QtPrivate::compareStrings performs UTF-16 codepoint comparisons
not Unicode character ones, so surrogate pairs were sorting before
U+E000 to U+FFFF.
To fix all of this, we need to decode the UTF-16 string into UTF-32 and
calculate the length of that in UTF-8 to be sure we have the sorting
order right.
Since this is a slight behavior change with a performance penalty, I am
choosing to backport only to 6.7. The penalty mostly does not apply to
6.8 due to commit 61556627f25e7c7acbfcc5e54127a392b5239977.
Change-Id: If1bf59ecbe014b569ba1fffd17c4c4ddcc874aac
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
(cherry picked from commit d4c7da9a07dc1434692fe08a61ba22c794574c4f)
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The lengths are only needed to check if one of the two strings is empty
and in the direct 8-bit comparisons with memcmp(). So we don't need this
division by 2, which was here since the initial commit.
Amends d4c7da9a07dc1434692fe08a61ba22c794574c4f.
Change-Id: Ie28eadac333c4bcd8c08fffd17c5ecbce564ccd1
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
(cherry picked from commit a7b9ec7f6f465ce0d90edfe7154a4837b1fb9c00)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
We must compare the raw string length in UTF-8, not the UTF-16 one.
[ChangeLog][QtCore][QCborValue] Fixed a bug that caused certain
non-US-ASCII string comparisons to produce results not in line with the
CBOR specifications.
Pick-to: 6.6 6.5
Change-Id: I5f663c2f9f4149af84fefffd17c05d1c0f1bbc3a
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
(cherry picked from commit 394788c68efacdec2676988b4b4ff207b20557f2)
... and use it in QCborContainerPrivate::replaceAt_complex() to avoid
unconstrained memory growth in certain scenarios.
Remove the `reserved` parameter, because it was referring to the
elements array, not to the byte data, so it cannot really be used
in the implementation.
Fixes: QTBUG-109073
Change-Id: I2e8fe7e4a4bf7a0ce06c87ca657f2bc01bae0341
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
(cherry picked from commit 83f0796192d9f0ed3bc005dbcf68b98de62955b8)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
The old name didn't say what sort of thing it was converting; and our
guidelines for examples discourage using the word Example in the
title. Also reword the description of the tool in the command-line
parser and an assertion.
Pick-to: 6.6 6.5
Task-number: QTBUG-111228
Change-Id: I6f52f5227362b4b807c8aabfd2103287af42bca0
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Partly because it also saves to CBOR, but also because our guidelines
say to avoid using "Example" in the title.
Pick-to: 6.6 6.5
Task-number: QTBUG-111228
Change-Id: Id858475a6b0474228cfe8044e188cc763f56e3a8
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
When converting from an array to a map, we double the number of
elements, spread the old elements out to make one free slot of space
in front of each, and then place Integer values counting from 0 into
the free slots.
The old code contained a loop that would add a strong reference to the
original elements that happen to be containers and thus are
ref-counted in the first place. But this additional strong reference
is not needed: In both cases, detached or in-place, the detach() call
that ensured unique ownership of 'map/dst' will have either directly
or indirectly updated the ref-counts of the elements correctly, and
the following loops just reshuffle the elements in the QList, they
don't create new copies (QtCbor::Elements doesn't by itself manipulate
container->ref, but even if it did, the copy would have increased, and
the assignment of the Integers would have decreased, the ref-count
again). Adding the strong ref without a user then caused the container
members to be leaked.
Fix by removing the loop.
[ChangeLog][QtCore][QCborValue] Fixed a memory leak when an array was
coerced into a map.
Amends ccea34464075759424e61806c7bc98ee3e658670.
Not picking to 6.4 as it's closed at this time.
Fixes: QTBUG-115249
Pick-to: 6.6 6.5
Change-Id: I369c372e91c3f0cfe3c65f9b0ea8507d08fdaf48
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
These APIs started out as private APIs in qnumeric_p.h, but have since
been made pseudo-public in qnumeric.h. The qnumeric_p.h versions just
forward to the qnumeric.h ones, so just use the latter.
This is in preparation of removing the {add,sub,mul}_overflow
versions, which, despite being defined in the unnamed namespace, don't
sport the q prefix, so potentially clash with global symbols.
The change is a simple textual search and replace, manually excluding
qnumeric_p.h.
Picking to 6.5 to avoid cherry-pick conflicts going forward.
Pick-to: 6.6 6.5
Change-Id: Ic0f7c92f7c47923317109e8a9dc06fa66bdff2c2
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Drive-by change: use QByteArrayView instead of allocating a QByteArray.
Change-Id: Iaf7acbbdb4efbb101b73b30061ce38dd1fa99ca3
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Unify wording as "{Latin-1,UTF-16} string viewed by \a str".
Drive-by change: Fix a grammatical error, it's "a US-ASCII", not an
(because it's pronounced by the letter name "U" which is pronounced
like "you", so "a" not "an").
Task-number: QTBUG-108711
Pick-to: 6.5
Change-Id: Iff763f4008341c35317bb3d7a2a228767ff6a648
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
This saves (mostly in corelib/time/) some complications that used to
arise from needing different code-paths for different time-specs.
Task-number: QTBUG-108199
Change-Id: I5dbd09859fce7599f1ba761f8a0bfc4633d0bef9
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Added some background about Serialization with the classes and used cases.
Fixes: QTBUG-103951
Pick-to: 6.4 6.3 6.2
Change-Id: I3ff179b814fc5d424f2ac2ffaf3237b90ddd7e2b
Reviewed-by: Vladimir Minenko <vladimir.minenko@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Kai Köhne <kai.koehne@qt.io>
We've been requiring C++17 since Qt 6.0, and our qAsConst use finally
starts to bother us (QTBUG-99313), so time to port away from it
now.
Since qAsConst has exactly the same semantics as std::as_const (down
to rvalue treatment, constexpr'ness and noexcept'ness), there's really
nothing more to it than a global search-and-replace, with manual
unstaging of the actual definition and documentation in dist/,
src/corelib/doc/ and src/corelib/global/.
Task-number: QTBUG-99313
Change-Id: I4c7114444a325ad4e62d0fcbfd347d2bbfb21541
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Add documentation of usage of CBOR in convert and cbordump examples,
add a CBOR overview, and add links to them other places in the
documentation.
Task-number: QTBUG-85912
Change-Id: I518792db63647bf9ddd4507d8d4b7ef056192f82
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
All of those are implicitly-shared Qt data types whose copy constructors
can't throw and have wide contracts (there aren't even any assertions
for validity in any of them). These are all types with a QVariant
implicit constructor, except for QCborValue, which is updated on this
list so QJsonValue (which has a QVariant constructor) is also
legitimately noexcept.
To ensure we haven't made a mistake, the Private constructor checks
again.
Change-Id: I3859764fed084846bcb0fffd17044d8319a45e1f
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
It's not used and not useful.
The macro itself has to stay, for now, because Qt5Compat uses it, too.
Task-number: QTBUG-100861
Pick-to: 6.4 6.3 6.2
Change-Id: I5d0557a6c959d6facf6e47f26786a9d365339e95
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Holding on to the memory we've just allocated could only make sense if
we supported resuming parsing, but I haven't implemented that feature
yet.
Pick-to: 6.2 6.3 6.4
Change-Id: I36b24183fbd041179f2ffffd1700cf7e3558040f
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
We don't promise we're very good at handling exceptions.
Pick-to: 6.4
Task-number: QTBUG-104718
Change-Id: I89c4eb48af38408daa7cfffd16fdcc171db437bd
Reviewed-by: Samuel Gaist <samuel.gaist@idiap.ch>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
We were allowing up to 1 million elements for each array, which meant 16
MB (32 for maps), and we considered that sufficient. However, because we
do allow up to 1024 levels of recursion, the memory consumption was
actually limited to 16 GB (32 for maps), which is a bit too high for
64-bit applications, and definitely too high for 32-bit ones.
So further clamp down, to a mere 16k elements on 32-bit and 64k on 64-
bit, which limits the memory use to 256 MB on 32-bit and 1 GB on 64-bit.
And additionally apply this exact limit to maps, instead of allowing
them to double the size.
As before, this does not limit the size of non-corrupt streams. This
only limits the initial memory pre-allocation.
Pick-to: 6.2 6.3 6.4
Fixes: QTBUG-104718
Change-Id: I89c4eb48af38408daa7cfffd16fdcb34f08c1949
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
We don't need them.
Drive-by opportunity to ensure byteData() didn't return null. It could
happen.
Change-Id: I74249c52dc02478ba93cfffd16d1e1fc060cdaef
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Yes, this is a cold function, but we can do it in-place if we want to.
Done in three separate passes so the autovectorizer can do its thing, if
it wants to.
Change-Id: I5e52dc5b093c43a3b678fffd16b6eb34d02fc40b
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The initial implementation from Qt 5.13 simply created a QCborValue and
then called the equivalent function on that result. For integrals and
double where the QCborValue matched the asked type, the cost was minimal
anyway. For the other types and when the type didn't match, this
resulted in up/down the reference counter in QCborContainerPrivate and
an out-of-line call to the destructor.
This improves the performance for code like:
for (QCborValue v : array) {
if (v.toString() == x) {
doSomething();
}
}
Change-Id: I5e52dc5b093c43a3b678fffd16b6022da4e2d446
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The 0x10000 limit should not apply if the key is a valid index in the
array.
Change-Id: I5e52dc5b093c43a3b678fffd16b6a2a5a69acd61
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
I screwed up when I wrote QCborValueRef by not having the ConstRef
type. The code worked, but it wasn't const-correct, allowing you to
write:
const QCborArray &arr = something();
*arr.begin() = QCborArray();
This mistake was brought over to QJsonValue in Qt 6.0, so it has to be
fixed for QJsonValue too. The actual fixes are in the next couple of
commits.
This change is believed to be binary-compatible: the Q{Json,Cbor}ValueRef
classes continue to have the exact same size, except that they're now
empty and have a new base class. They weren't trivial before this commit
doesn't change that.
[ChangeLog][Potentially Source-Incompatible Changes] The iterator
classes for Qt's JSON and CBOR containers (array and map/object) had a
const correctness issue which allowed a const_iterator to mutate the
container being iterated on, even if that container was itself const. Qt
6.4 has a fix for this, but will cause compilation issues where
QCborValueRef and QJsonValueRef were used where the correctness could be
violated. To keep code compiling with both 6.3 and 6.4, either change to
non-const iteration or replace the QxxxValueRef with a const QxxxValue
reference. This change is binary-compatible.
Change-Id: I5e52dc5b093c43a3b678fffd16b6063333765ae0
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
The flag IsContainer was not set, causing the QCborContainerPrivate to
become confused.
This commit also expands and subsumes the existing test for QCborValue
(non-Ref).
Change-Id: I5e52dc5b093c43a3b678fffd16b6a17c6f4a0676
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
This has found several missing const qualifications, a missing
QCborMap::Iterator method, and a missing one in QCborValue too.
The methods "### TEMPORARY" in this commit are actually removed in two
commits.
Change-Id: I5e52dc5b093c43a3b678fffd16b6939f62954dc4
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
When QCborValue referred to an empty array or map, toArray() and toMap()
would respectively return the default value instead of the empty object,
as expected.
Pick-to: 6.2 6.3
Change-Id: I5e52dc5b093c43a3b678fffd16b60456d0037ad7
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
In the effort of repairing broken links as per QTBUG-96127,
a series of RFC links referring to `tools.ietf.org/html/*` were modified
to point to the new address that the site redirected to.
To simplify executing a similar task and to diminish the duplication of
manually inserted urls, the already existing `rfc.qdoc` file, containing
`\externalpage` commands directing to RFC locations, was enhanced with
links to all RFCs that were mentioned in the current documentation, so
as to aggregate this common category of links.
All links pointing to a `ietf` domain inside QDoc documentation blocks
were then changed to use the newly provided external-references.
Task-number: QTBUG-96127
Pick-to: 6.2
Change-Id: I2a52eb6aa8c9e346f64ef1a627b039220d9f6c2a
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This patch is inspired by CodeChecker, that is complaining about
possible nullptr dereferencing.
Currently it is a false-positive, because the codepath suggested by
CodeChecker is impossible with current implementation.
But having asserts can save some time in case of possible refactoring.
Task-number: QTBUG-95727
Pick-to: 6.2
Change-Id: I242a23e8aaa249cce16b867c0884dfc3849977f5
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
One was only called once. For the other, one of the calls wasn't
necessary.
Pick-to: 5.15 6.0
Change-Id: I7b9b97ae9b32412abdc6fffd16454eec59d72b8a
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Similar to the previous commit which applied to QCborStreamReader, don't
allocate too much data before checking that the stream actually has that
much.
Pick-to: 5.15 6.0
Fixes: QTBUG-88256
Change-Id: I7b9b97ae9b32412abdc6fffd16454b7568a063ba
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Try to get rid of APIs that use raw 'const {char, QChar} *, length'
pairs. Instead, use QByteArrayView or QStringView.
As QStringConverter is a new class, simply change the API to what we'd like
to have. Also adjust hidden API in QStringBuilder and friends.
Change-Id: I897d47f63a7b965f5574a1e51da64147f9e981f6
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Change-Id: Ibdc95e9af7bd456a94ecfffd16066c47ea9766d0
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Omitting state machine and docs for now.
Task-number: QTBUG-84469
Change-Id: Ibfa5e7035515773461f6cdbff35299315ef65737
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
We checked against integer overflow, but not against overflowing the
QByteArray size limit. That caused a std::bad_alloc to be thrown, which
is bad when decoding unknown data. QCborStreamReader wasn't affected,
since it doesn't merge chunks.
Change-Id: I99ab0f318b1c43b89888fffd160c36f495fada87
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
