1berry/qtbase
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Correct handling of World in mapping MS's zone IDs to IANA ones

Browse Source 
The AnyTerritory entries in the zoneDataTable are derived from
territory="ZZ" entries in the upstream CLDR data; the World ones from
territory="001". The latter give the default IANA ID for each MS ID,
the former give an (often legacy) IANA ID for the MS ID, that is not
based on geography. Some of these are being removed at CLDR v46.

The documentation said the ZZ entries have "no known territorial
association", hinting that there may be some (unknown) territorial
association; however, CLDR's inclusion of them is as entries with a
known non-territorial association, so revise the phrasing to reflect
this.

Also document that windowsIdToDefaultIanaId() returns empty when
there is no territory-specific value, and callers can use the
territory-neutral call to get a suitable value in that case. (They
may, however, wish to distinguish this case, to treat it differently,
so I decided not to just return that in place of empty in any case.)

The upstream CLDR tables do have entries for territory 001, so we
should report these if asked for World as territory. Amend the
available zone ID lookup and mapping from MS to IANA functions that
take a territory to duly handle World via the default-data that was
derived from 001 data in CLDR, instead of from the territory-varying
table, from which those were effectively filtered out when generating
the two tables. Update docs to mention this handling of World, for
contrast with that of AnyTerritory.

In the process remove a spurious split-on-space from the MS to default
IANA lookup, asserting there is no space (in a field now stored in the
table for single IANA ID entries, instead of the one for space-joined
lists of them in which it used to be stored, before I noticed it's
always only one ID). There is a matching assertion in the cldr.py code
that extracts the data. Added an assertion to this last, that each
default IANA ID given by CLDR's MS data does in fact also appear as
one of the IANA IDs for at least one territory (potentially ZZ), and
comment in C++ code on why this means we don't need to scan the
windowsDataTable in a few places, where it would just produce
duplicate entries.

[ChangeLog][QtCore][QTimeZone] Corrected handling of QLocale::World
and clarified in docs how QLocale::AnyTerritory is handled when
QTimeZone selects zones by territory.

Pick-to: 6.8
Task-number: QTBUG-130877
Change-Id: I861c777c68b0cb73a194138fe23fbff839df49e6
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This commit is contained in:
Edward Welbourne 2024-11-29 13:17:21 +01:00
parent 79a74aa768
commit e23dc7c420
4 changed files with 73 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
src/corelib/time/qtimezone.cpp
View File

@ -1529,10 +1529,11 @@ QList<QByteArray> QTimeZone::availableTimeZoneIds()
/*!
Returns a list of all available IANA time zone IDs for a given \a territory.
As a special case, a \a territory of \l {QLocale::}{AnyTerritory} selects
those time zones that have no known territorial association, such as UTC. If
you require a list of all time zone IDs for all territories then use the
standard availableTimeZoneIds() method.
As a special case, a \a territory of \l {QLocale::} {AnyTerritory} selects
those time zones that have a non-territorial association, such as UTC, while
\l {QLocale::}{World} selects those time-zones for which there is a global
default IANA ID. If you require a list of all time zone IDs for all
territories then use the standard availableTimeZoneIds() method.
This method is only available when feature \c timezone is enabled.
@ -1601,8 +1602,14 @@ QByteArray QTimeZone::windowsIdToDefaultIanaId(const QByteArray &windowsId)
Because a Windows ID can cover several IANA IDs within a given territory,
the most frequently used IANA ID in that territory is returned.
As a special case, \l{QLocale::}{AnyTerritory} returns the default of those
IANA IDs that have no known territorial association.
As a special case, \l {QLocale::} {AnyTerritory} returns the default of
those IANA IDs that have a non-territorial association, while \l {QLocale::}
{World} returns the default for the given \a windowsId in territories that
have no specific association with it.
If the return is empty, there is no IANA ID specific to the given \a
territory for this \a windowsId. It is reasonable, in this case, to fall
back to \c{windowsIdToDefaultIanaId(windowsId)}.
This method is only available when feature \c timezone is enabled.
@ -1633,8 +1640,10 @@ QList<QByteArray> QTimeZone::windowsIdToIanaIds(const QByteArray &windowsId)
/*!
Returns all the IANA IDs for a given \a windowsId and \a territory.
As a special case, \l{QLocale::}{AnyTerritory} selects those IANA IDs that
have no known territorial association.
As a special case, \l{QLocale::} {AnyTerritory} selects those IANA IDs that
have a non-territorial association, while \l {QLocale::} {World} selects the
default for the given \a windowsId in territories that have no specific
association with it.
The returned list is in order of frequency of usage, i.e. larger zones
within a territory are listed first.

48
src/corelib/time/qtimezoneprivate.cpp
View File

@ -630,10 +630,17 @@ QList<QByteArray> QTimeZonePrivate::availableTimeZoneIds(QLocale::Territory terr
regions = QtTimeZoneLocale::ianaIdsForTerritory(territory);
#endif
// Get all Zones in the table associated with this territory:
for (const ZoneData &data : zoneDataTable) {
if (data.territory == territory) {
for (auto l1 : data.ids())
regions << QByteArrayView(l1.data(), l1.size());
if (territory == QLocale::World) {
// World names are filtered out of zoneDataTable to provide the defaults
// in windowsDataTable.
for (const WindowsData &data : windowsDataTable)
regions << data.ianaId();
} else {
for (const ZoneData &data : zoneDataTable) {
if (data.territory == territory) {
for (auto l1 : data.ids())
regions << QByteArrayView(l1.data(), l1.size());
}
}
}
return selectAvailable(std::move(regions), availableTimeZoneIds());
@ -803,6 +810,8 @@ QByteArray QTimeZonePrivate::ianaIdToWindowsId(const QByteArray &id)
return toWindowsIdLiteral(data.windowsIdKey);
}
}
// If the IANA ID is the default for any Windows ID, it has already shown up
// as an ID for it in some territory; no need to search windowsDataTable[].
return QByteArray();
}
@ -812,8 +821,7 @@ QByteArray QTimeZonePrivate::windowsIdToDefaultIanaId(const QByteArray &windowsI
windowsId, earlierWindowsId);
if (data != std::end(windowsDataTable) && data->windowsId() == windowsId) {
QByteArrayView id = data->ianaId();
if (qsizetype cut = id.indexOf(' '); cut >= 0)
id = id.first(cut);
Q_ASSERT(id.indexOf(' ') == -1);
return id.toByteArray();
}
return QByteArray();
@ -837,6 +845,9 @@ QList<QByteArray> QTimeZonePrivate::windowsIdToIanaIds(const QByteArray &windows
for (auto l1 : data->ids())
list << QByteArray(l1.data(), l1.size());
}
// The default, windowsIdToDefaultIanaId(windowsId), is always an entry for
// at least one territory: cldr.py asserts this, in readWindowsTimeZones().
// So we don't need to add it here.
// Return the full list in alpha order
std::sort(list.begin(), list.end());
@ -847,16 +858,21 @@ QList<QByteArray> QTimeZonePrivate::windowsIdToIanaIds(const QByteArray &windows
QLocale::Territory territory)
{
QList<QByteArray> list;
const quint16 windowsIdKey = toWindowsIdKey(windowsId);
const qint16 land = static_cast<quint16>(territory);
for (auto data = zoneStartForWindowsId(windowsIdKey);
data != std::end(zoneDataTable) && data->windowsIdKey == windowsIdKey;
++data) {
// Return the region matches in preference order
if (data->territory == land) {
for (auto l1 : data->ids())
list << QByteArray(l1.data(), l1.size());
break;
if (territory == QLocale::World) {
// World data are in windowsDataTable, not zoneDataTable.
list << windowsIdToDefaultIanaId(windowsId);
} else {
const quint16 windowsIdKey = toWindowsIdKey(windowsId);
const qint16 land = static_cast<quint16>(territory);
for (auto data = zoneStartForWindowsId(windowsIdKey);
data != std::end(zoneDataTable) && data->windowsIdKey == windowsIdKey;
++data) {
// Return the region matches in preference order
if (data->territory == land) {
for (auto l1 : data->ids())
list << QByteArray(l1.data(), l1.size());
break;
}
}
}

23
tests/auto/corelib/time/qtimezone/tst_qtimezone.cpp
View File

@ -973,10 +973,16 @@ void tst_QTimeZone::availableTimeZoneIds()
qDebug() << QTimeZone::availableTimeZoneIds(0);
qDebug() << "";
} else {
//Just test the calls work, we cannot know what any test machine has available
// Test the calls work:
QList<QByteArray> listAll = QTimeZone::availableTimeZoneIds();
QList<QByteArray> listUs = QTimeZone::availableTimeZoneIds(QLocale::UnitedStates);
QList<QByteArray> listZero = QTimeZone::availableTimeZoneIds(0);
QList<QByteArray> list001 = QTimeZone::availableTimeZoneIds(QLocale::World);
QList<QByteArray> listUsa = QTimeZone::availableTimeZoneIds(QLocale::UnitedStates);
QList<QByteArray> listGmt = QTimeZone::availableTimeZoneIds(0);
// We cannot know what any test machine has available, so can't test contents.
// But we can do a consistency check:
QCOMPARE_LT(list001.size(), listAll.size());
QCOMPARE_LT(listUsa.size(), listAll.size());
QCOMPARE_LT(listGmt.size(), listAll.size());
}
}
@ -1034,9 +1040,9 @@ void tst_QTimeZone::windowsId()
/*
Current Windows zones for "Central Standard Time":
Region IANA Id(s)
Default "America/Chicago"
World "America/Chicago" (the default)
Canada "America/Winnipeg America/Rankin_Inlet America/Resolute"
Mexico "America/Matamoros"
Mexico "America/Matamoros America/Ojinaga"
USA "America/Chicago America/Indiana/Knox America/Indiana/Tell_City America/Menominee"
"America/North_Dakota/Beulah America/North_Dakota/Center"
"America/North_Dakota/New_Salem"
@ -1055,6 +1061,8 @@ void tst_QTimeZone::windowsId()
// Check default value
QCOMPARE(QTimeZone::windowsIdToDefaultIanaId("Central Standard Time"),
QByteArray("America/Chicago"));
QCOMPARE(QTimeZone::windowsIdToDefaultIanaId("Central Standard Time", QLocale::World),
QByteArray("America/Chicago"));
QCOMPARE(QTimeZone::windowsIdToDefaultIanaId("Central Standard Time", QLocale::Canada),
QByteArray("America/Winnipeg"));
QCOMPARE(QTimeZone::windowsIdToDefaultIanaId("Central Standard Time", QLocale::AnyTerritory),
@ -1072,6 +1080,11 @@ void tst_QTimeZone::windowsId()
};
QCOMPARE(QTimeZone::windowsIdToIanaIds("Central Standard Time"), list);
}
{
const QList<QByteArray> list = { "America/Chicago" };
QCOMPARE(QTimeZone::windowsIdToIanaIds("Central Standard Time", QLocale::World),
list);
}
{
// Check country with no match returns empty list
const QList<QByteArray> empty;

6
util/locale_database/cldr.py
View File

@ -620,6 +620,12 @@ enumdata.py (keeping the old name as an alias):
else:
windows.append((wid, code, ' '.join(ianas)))
# For each Windows ID, its default zone is its zone for at
# least some territory:
assert all(any(True for w, code, seq in windows
if w == wid and zone in seq.split())
for wid, zone in defaults.items()), (defaults, windows)
return defaults, windows
def readMetaZoneMap(self, alias: dict[str, str]