John Layt
2011-10-20 20:36:52 UTC
I have now completed most of the Calendar System framework and think it's
close to ready for an initial review. The code is at the usual place:
https://qt.gitorious.org/~odysseus/qt/odysseus-qtbase/commits/qt5-qdatetime
Current changes:
1) Various small cleanups
2) QDate stores JD as a qint64
3) Separate QDateTimeFormatter and QDateTimeParser classes
4) QDateCalculator class to calculate calendar systems
5) QLocale uses CLDR date/time formatting [small source incompatible
changes]
6) Modify QDateTime* to use the CLDR data and QDateCalculator via QLocale
There's still a few areas to be completed:
1) More documentation and unit tests
3) Hebrew, Islamic Lunar, Chinese Lunar calendars
4) Era support
5) Localized week number support
6) New lightweight datetime parser
7) New or improved widgets
8) Modify QDate to make old code QT4_COMPAT
9) Convenience wrapper class, i.e. QLocaleDate
10) Full Windows and OSX integration
These are mostly polish and fit I can work on later, but there is one further
area that is probably a show-stopper now, and that is memory use. Currently
Qt compiles all the locale data into the QtCore library itself as static const
for speed and convenience at the cost of some memory, but adding calendar
systems bloats the memory use right out:
Qt4.8 locale data size 700 kB
Qt5 with QT_NO_CALENDAR_SYSTEMS 750 kB
Qt5 with Calendar support 3.5 MB
The actual calendar data such as names and translations is only 125kB more,
it's the index into that data for each locale that adds 2.7MB. This doesn't
yet include the Era or Time Zone translations that I want to use, so this is
only going to get bigger.
Just for comparison, all the CLDR data in ICU including conversion tables
takes 15 MB and can be either compiled into the library or loaded from file,
see http://userguide.icu-project.org/icudata for details.
I think I can do a more efficient index and reduce size by 30-50%, but that's
still bloated and really complicates the backend code, so I may need to look
at a file based alternative.
Denis mentioned at QtCS that the embedded guys were wanting to reduce
the memory footprint anyway and to be able to choose what locales they ship,
so it could be an all-round win to change to file loading.
I can think of a few options:
1) Offload just the translation data into translation files and use tr()
2) Offload all the data into binary files keeping the current data structure
3) Offload all the data into a single binary file (use ICU .dat format?)
4) Offload all the data into one binary file per locale
Option 1 may not work due to dependencies. Option 2 is close to the current
implementation, but puts choosing locales at the file generation stage, as
does option 3. Option 4 seems the most flexible but will probably take a lot
more disk space than 2 or 3 due to data duplication. Any option probably
means changing from static const data to QSharedData.
Any comments or suggestions?
John.
close to ready for an initial review. The code is at the usual place:
https://qt.gitorious.org/~odysseus/qt/odysseus-qtbase/commits/qt5-qdatetime
Current changes:
1) Various small cleanups
2) QDate stores JD as a qint64
3) Separate QDateTimeFormatter and QDateTimeParser classes
4) QDateCalculator class to calculate calendar systems
5) QLocale uses CLDR date/time formatting [small source incompatible
changes]
6) Modify QDateTime* to use the CLDR data and QDateCalculator via QLocale
There's still a few areas to be completed:
1) More documentation and unit tests
3) Hebrew, Islamic Lunar, Chinese Lunar calendars
4) Era support
5) Localized week number support
6) New lightweight datetime parser
7) New or improved widgets
8) Modify QDate to make old code QT4_COMPAT
9) Convenience wrapper class, i.e. QLocaleDate
10) Full Windows and OSX integration
These are mostly polish and fit I can work on later, but there is one further
area that is probably a show-stopper now, and that is memory use. Currently
Qt compiles all the locale data into the QtCore library itself as static const
for speed and convenience at the cost of some memory, but adding calendar
systems bloats the memory use right out:
Qt4.8 locale data size 700 kB
Qt5 with QT_NO_CALENDAR_SYSTEMS 750 kB
Qt5 with Calendar support 3.5 MB
The actual calendar data such as names and translations is only 125kB more,
it's the index into that data for each locale that adds 2.7MB. This doesn't
yet include the Era or Time Zone translations that I want to use, so this is
only going to get bigger.
Just for comparison, all the CLDR data in ICU including conversion tables
takes 15 MB and can be either compiled into the library or loaded from file,
see http://userguide.icu-project.org/icudata for details.
I think I can do a more efficient index and reduce size by 30-50%, but that's
still bloated and really complicates the backend code, so I may need to look
at a file based alternative.
Denis mentioned at QtCS that the embedded guys were wanting to reduce
the memory footprint anyway and to be able to choose what locales they ship,
so it could be an all-round win to change to file loading.
I can think of a few options:
1) Offload just the translation data into translation files and use tr()
2) Offload all the data into binary files keeping the current data structure
3) Offload all the data into a single binary file (use ICU .dat format?)
4) Offload all the data into one binary file per locale
Option 1 may not work due to dependencies. Option 2 is close to the current
implementation, but puts choosing locales at the file generation stage, as
does option 3. Option 4 seems the most flexible but will probably take a lot
more disk space than 2 or 3 due to data duplication. Any option probably
means changing from static const data to QSharedData.
Any comments or suggestions?
John.