JSON (was Re: Some parts of LibQxt in Qt)

Post by Girish Ramakrishnan
Does anyone have a strong opinion on what API we should have for a
JSON parser in Qt? I quite like the qjson approach of just providing a
QVariant. QVariant conversion has a performance overhead but it's very

all qt json libs I saw use QVariant and it works very nice. JSON is a network protocol so its critical section is the network speed not a QVariant conversions.conrtsuctors.

I'd like to see json in the QtCore

Post by Girish Ramakrishnan
easy to work with and from what I have seen the big performance impact
is in the parsing phase and not when getting data out of the QVariant.
The alternate is a stream based API like the XML one but I think this
is just too cumbersome.

yes, XML-like streams are overkill in this area.

Petr Vanek

2011-09-27 21:26:06 UTC

heh, not exactly qtcore, but something like "core" qtnetwork... sorry.

Stefan Majewsky

2011-09-28 08:47:47 UTC

Post by Petr Vanek
I'd like to see json in the QtCore

heh, not exactly qtcore, but something like "core" qtnetwork... sorry.

I don't see why JSON must be restricted to QtNetwork. It's very
valuable as a simple, yet versatile in-house data specification
format. For example, I've already used it as an input format in a
simple C++ code generator. Like Girish said, an implementation can be
done in a couple hundred lines, so not something I'd like to pull in
complete QtNetwork for.

Greetings
Stefan

s***@accenture.com

2011-09-28 09:39:13 UTC

-----Original Message-----
On Behalf Of Stefan Majewsky
Sent: Wednesday, September 28, 2011 09:48
Subject: Re: [Qt5-feedback] JSON (was Re: Some parts of LibQxt in Qt)

Post by Petr Vanek
I'd like to see json in the QtCore

heh, not exactly qtcore, but something like "core" qtnetwork...

sorry.
I don't see why JSON must be restricted to QtNetwork. It's very
valuable as a simple, yet versatile in-house data specification
format. For example, I've already used it as an input format in a
simple C++ code generator. Like Girish said, an implementation can be
done in a couple hundred lines, so not something I'd like to pull in
complete QtNetwork for.

Is there any reason not to have json as a standalone library then?
A web services type of application could link against core, network and json

Arguments against would be if the DLL overhead is too high compared to the code / data sizes.
(e.g. the 4 bytes of static data rounded up to a 4k page per process problem)

________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.

Girish Ramakrishnan

2011-09-28 10:04:51 UTC

Post by s***@accenture.com

-----Original Message-----
On Behalf Of Stefan Majewsky
Sent: Wednesday, September 28, 2011 09:48
Subject: Re: [Qt5-feedback] JSON (was Re: Some parts of LibQxt in Qt)

Post by Petr Vanek
I'd like to see json in the QtCore

heh, not exactly qtcore, but something like "core" qtnetwork...

Is there any reason not to have json as a standalone library then?
A web services type of application could link against core, network and json

The json code is really small, so it's a bit of an overkill to make it
a separate library. We should just place it alongside the XML stream
parser in QtCore.

Girish

Thiago Macieira

2011-09-28 15:09:09 UTC

Post by s***@accenture.com

Post by Stefan Majewsky
I don't see why JSON must be restricted to QtNetwork. It's very
valuable as a simple, yet versatile in-house data specification
format. For example, I've already used it as an input format in a
simple C++ code generator. Like Girish said, an implementation can be
done in a couple hundred lines, so not something I'd like to pull in
complete QtNetwork for.

Is there any reason not to have json as a standalone library then?

Other than the reason you found below, I don't think there's any.

Post by s***@accenture.com
Arguments against would be if the DLL overhead is too high compared to the
code / data sizes. (e.g. the 4 bytes of static data rounded up to a 4k page
per process problem)

You used a Symbian symptom to the problem of having too many libraries, but
other OSes have similar issues. On Linux, the symbol search complexity is
O(n*m), which depends on the number of libraries loaded (one hashing table per
library with a bad hashing function).

If this codebase turns out to be small, flexible and fast, I'd vote for having
it in QtCore. We have QXmlStreamReader and Writer there for a reason, moved
out of QtXml.

--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

Jedrzej Nowacki

2011-10-04 07:20:46 UTC

Post by Girish Ramakrishnan
Hi,

Hi,

Here's one I wrote based on QLALR and it's way faster than the (very
http://gitorious.org/qjsonparser/qjsonparser.
Does V8 have it's own json parser that we can use?

That's why I said we have 5 different JSON parsers today and Darwin
should determine which one we'll pick.

About V8, Qt is gonna use that right? I'm guessing V8 has it's own uber
fast json parser so why not make a qt layer for that..? Seems like that
gives no duplication and json in c++, everyone happy :)

AFAICT, v8's json parser is written in JavaScript. I don't think it's
a problem that it's not in C++, though. What I do think is a problem
is that apps now need to have v8 just to use JSON (maybe this is an
imaginary problem).
If I understood Thiago, he would prefer to 'wait and see which one
survives'. I would instead like us to choose one and provide an api
for JSON in qtbase, asap. Pretty much every project I work with needs
JSON these days. JSON implementation is quite trivial anyway (like
~200-300 lines max), so I don't see the need to have competing
implementations 'fight it out'.
Does anyone have a strong opinion on what API we should have for a
JSON parser in Qt? I quite like the qjson approach of just providing a
QVariant. QVariant conversion has a performance overhead but it's very
easy to work with and from what I have seen the big performance impact
is in the parsing phase and not when getting data out of the QVariant.
The alternate is a stream based API like the XML one but I think this
is just too cumbersome.
Girish
_______________________________________________
Qt5-feedback mailing list
http://lists.qt.nokia.com/mailman/listinfo/qt5-feedback

Hi,

As Thiago already pointed out, we have few (too many) competing
implementation of JSON parsers, and two different environments (C++ and JS/QML)
in which they should work. We should have only one implementation mainly
because we can't allow behavior differences and because of general maintenance
costs (testing, performance improvements, etc.).

Currently, I do not see obvious winner. For QML, V8 implementation may be
preferred, as it creates a native JS object, which can be used directly from a
QML application. On other hand from C++ point of view it would be an overkill
to start full JS environment to parse a tiny json file. But of course everyone
will use QML, so it is not a problem, right :-)? What I do not like about
qjsonparser mentioned by Girish, is that it uses QVariant as a type for object
storage. It won't be fast, ever. QVariant is a type safe replacement for
void*, and nothing more. I think it is misuse to treat it as a correct object
representation. In Qt, we already have two classes, which takes that
responsibilities; QObject and QJSValue. The first should be known to everyone,
second represents a JS object. Another issue related to QVariant is that it
always needs to be converted to something, in a common case you need to
iterate over QVariantMap to create an useful representation.

I think we need more benchmarks, before we can make the right decision.

Cheers,
JÄdrek

ps. JSON means JavaScript Object Notation, so JS and other dynamic languages
will take most of it, for C++ it will be always a bit artificial.

Konstantin Tokarev

2011-10-04 07:30:19 UTC

ps. JSON means JavaScript Object Notation, so JS and other dynamic languages will take most of it, for C++ it will be always a bit artificial.

No more than XML is.

--
Regards,
Konstantin

Girish Ramakrishnan

2011-10-05 13:51:23 UTC

Hi Jedrzej,

On Tue, Oct 4, 2011 at 12:50 PM, Jedrzej Nowacki

Post by Jedrzej Nowacki
Hi,
As Thiago already pointed out, we have few (too many) competing
implementation of JSON parsers, and two different environments (C++ and
JS/QML) in which they should work. We should have only one implementation
mainly because we can't allow behavior differences and because of general
maintenance costs (testing, performance improvements, etc.).
Currently, I do not see obvious winner. For QML, V8 implementation may be
preferred, as it creates a native JS object, which can be used directly from
a QML application. On other hand from C++ point of view it would be an
overkill to start full JS environment to parse a tiny json file. But of
course everyone will use QML, so it is not a problem, right :-)? What I do
not like about qjsonparser mentioned by Girish, is that it uses QVariant as
a type for object storage. It won't be fast, ever. QVariant is a type safe
replacement for void*, and nothing more. I think it is misuse to treat it as
a correct object representation. In Qt, we already have two classes, which
takes that responsibilities; QObject and QJSValue. The first should be known
to everyone, second represents a JS object. Another issue related to
QVariant is that it always needs to be converted to something, in a common
case you need to iterate over QVariantMap to create an useful
representation.

Right, we are sort of "reusing" QVariant to store the entire json
structure since it incidentally happens to support all the types that
appear in json :) So, maybe we can make the return value be
JsonObject. You can then say:
JsonObject map = obj.map("foo");
int someInt = obj.intValue("bar");
JsonArray arr = obj.array("baz");

(After I wrote this, I see that this is basically very similar to
http://developer.android.com/reference/org/json/JSONObject.html)

If we can agree to the above, then I can create a MR.

Post by Jedrzej Nowacki
I think we need more benchmarks, before we can make the right decision.

I can benchmark my implementation against the others, if the owners
can first tell me that it is completely standards compliant (encoding
detection etc). I have benchmarked against qjson and it's way faster.
The benchmark results are in the qjsonparser repo.

Girish

Jedrzej Nowacki

2011-10-07 14:30:49 UTC

Post by Girish Ramakrishnan
Hi Jedrzej,
On Tue, Oct 4, 2011 at 12:50 PM, Jedrzej Nowacki

Post by Jedrzej Nowacki
Hi,
As Thiago already pointed out, we have few (too many) competing
implementation of JSON parsers, and two different environments (C++ and
JS/QML) in which they should work. We should have only one implementation
mainly because we can't allow behavior differences and because of general
maintenance costs (testing, performance improvements, etc.).
Currently, I do not see obvious winner. For QML, V8 implementation may be
preferred, as it creates a native JS object, which can be used directly
from a QML application. On other hand from C++ point of view it would be
an overkill to start full JS environment to parse a tiny json file. But
of course everyone will use QML, so it is not a problem, right :-)? What
I do not like about qjsonparser mentioned by Girish, is that it uses
QVariant as a type for object storage. It won't be fast, ever. QVariant
is a type safe replacement for void*, and nothing more. I think it is
misuse to treat it as a correct object representation. In Qt, we already
have two classes, which takes that responsibilities; QObject and
QJSValue. The first should be known to everyone, second represents a JS
object. Another issue related to QVariant is that it always needs to be
converted to something, in a common case you need to iterate over
QVariantMap to create an useful
representation.

Yes, something like that would be generic and have chance to be fast :-). On
top of it, we may create additional convenience function without much overhead
(like QJSValue or QVariantMap conversion).

Post by Jedrzej Nowacki
I think we need more benchmarks, before we can make the right decision.

I tried to benchmark it against v8 json parser. The benchmark is rather a
smoke test, then proper scientific measurement, but it seems that V8 is
significantly faster. Result and test in attachment. Important benchmark would
be to replace v8 json parser by a custom one and see if it doesn't hit
performance of QML.

Cheers,
JÄdrek

Mark

2011-10-10 08:35:53 UTC

On Fri, Oct 7, 2011 at 4:30 PM, Jedrzej Nowacki

Post by Girish Ramakrishnan
Hi Jedrzej,
On Tue, Oct 4, 2011 at 12:50 PM, Jedrzej Nowacki

implementation

Post by Jedrzej Nowacki
mainly because we can't allow behavior differences and because of

general

Post by Jedrzej Nowacki
maintenance costs (testing, performance improvements, etc.).
Currently, I do not see obvious winner. For QML, V8 implementation may

Post by Jedrzej Nowacki
preferred, as it creates a native JS object, which can be used directly
from a QML application. On other hand from C++ point of view it would

Post by Jedrzej Nowacki
an overkill to start full JS environment to parse a tiny json file. But
of course everyone will use QML, so it is not a problem, right :-)?

What

Post by Jedrzej Nowacki
I do not like about qjsonparser mentioned by Girish, is that it uses
QVariant as a type for object storage. It won't be fast, ever. QVariant
is a type safe replacement for void*, and nothing more. I think it is
misuse to treat it as a correct object representation. In Qt, we

already

Post by Jedrzej Nowacki
have two classes, which takes that responsibilities; QObject and
QJSValue. The first should be known to everyone, second represents a JS
object. Another issue related to QVariant is that it always needs to be
converted to something, in a common case you need to iterate over
QVariantMap to create an useful
representation.

Yes, something like that would be generic and have chance to be fast :-).
On top of it, we may create additional convenience function without much
overhead (like QJSValue or QVariantMap conversion).