You wrote, ‘”/” is illegal unescaped in a JSON string’. I’m looking at RFC 4627 which lists this ABNF rule:

unescaped = %x20-21 / %x23-5B / %x5D-10FFFF

‘/’ is %x2F, which falls in the second legal range for unescaped string characters. Similarly, http://www.json.org lists legal unescaped characters as:

any-Unicode-except-”-or-\-or-control

‘/’is neither ‘”‘ nor ‘\’ nor a control character, so it is also legal by this definition.

If you are using designated extension zones then why have a structured data format at all? Why not just have flat name/value pairs and use links for structure (ala RDF)? If structure matters then placing your extensions in the right place in the structure matters. Extension zones, by definition, allow extensions only in pre-approved places. So if the right place for your extension is somewhere other than the blessed extension zone then your SOL. I think you either have to pick a RDF style approach or you have to allow arbitrary extensibility, but you can’t have it both ways.

I actually am a big fan of the wrapper object approach. If you are changing something in a manner that isn’t backwards compatible (and therefore needs a must understand to point this out) then you should change the parent element because you are breaking the contract.

In a previous life, I did a fair amount of work with ASN.1 (including working on the spec for an ASN.1 based standard). In my original comment, I was proposing designated extensions zones for JSON.

if it didn’t violate my requirement of assuming nothing of the receiver, I’d also propose mustUnderstand for imported fields.

(I suppose you could kludge one by make a wrapper object in another namespace, adding your field, and making the original message a field on the wrapper.)

There are a couple of answer. My personal favorite is – ease of programming and debugging.

If you had asked me 11 or 12 years ago if self describing data was important I’m not sure I’d say yes. I’d probably say ‘naah, perf matters more, encode it, compress it, whatever, so long as your debugging tools can decompress it, who cares?’

Then HTTP happened. I have to admit that I wouldn’t have believed it if I hadn’t lived through it. But it turns out that having protocols and data formats that are trivial to produce and trivial to write code to consume and to debug on the fly (ahh the power of Telnet, and yes, I, like others, have debugged HTTP via Telnet, I’m probably just old though) makes it remarkably easy for people to adopt your protocol/data format.

By way of comparison lets look at something like EDI or even LDAP. Both use heavily schema’d binary protocols. The good news is that they are very efficient. The bad news is that you can’t use printf (or write or whatever your favorite text output is) to produce them, forget scanf for consuming them and lord help you if you ever need to debug them. Yes, you can get special tools that will decode them on the fly, if you have the schema. If you don’t have the schema (e.g. you are debugging a network and not sure what’s flying around) then you’re completely screwed, all you have is a collection of position based fields. Good luck.

Not being able to easily write, consume or debug a protocol or data format turns out to put a huge barrier to entry between the programmer and the format. Which is why I believe that HTTP and even XML took off while LDAP and EDI live in very special closed off communities. Of course given how much money goes over EDI maybe I should pray for failure but that’s a separate issue.

The other reason that self describing formats really, really matter is extensibility. Extensibility covers the stuff that isn’t in the schema. In theory you can fix this by designated extension zones. LDAP does something like this. They hacked their binary format to allow people to add in arbitrary properties. But only in certain places. Of course if the best place for your extension isn’t the place people thought would be best then you’re screwed. In theory this doesn’t matter since you can use links to hook things together but if ‘you can just link it together’ really worked we would all be using RDF. It turns out that location matters for understanding so you really want to be able to put your extensions next to the thing being extended. Not in some random location that someone decided extensions might make sense in.

And, here’s the best part, LDAP’s extension properties are self describing name/value pairs. Why self describing? Because it turns out that multiple different groups want to simultaneously expand the same entry. The schema is useless here because the groups aren’t coordinating so there is no central schema so without a self describing format (and some kind of name collision avoidance mechanism, hence namespaces) everyone ends up with an unreadable mess (bytes 15-30 are mine damnit! No, they are mine!). So if we ever want to be able to expand messages after they are released (and if the HTTP taught us anything it’s don’t paint yourself into a corner) then we need self describing formats, even when we have schemas.

In fact my own interest in schemas is primarily for marshaling and extensibility. Specifically, I want to use a schema so a programmer can say “This is the message I’m expecting and here’s how I want to rip it into memory” and I also want the programmer to be able to use the schema as a filter to rip out any extensions they don’t support so their code doesn’t blow up if they are sent a message with an extension that didn’t exist when the code was written.

So, to summarize (this really should be a blog article) I want a self describing format even when I have a schema because it makes it easier to write data, read data, debug data and extend data.

I guess I’m wondering why you want from a schema? What does it give you that “self-describing” doesn’t give you already? Are you trying to specify a contract? Are you trying to get your messages into a strongly typed language?

If you’re going to have a schema, why not skip the overhead of self-describing?

As for ‘imports’, this format would make it difficult to directly include fields from other namespaces. They would also be ‘children’ and forced off to a corner (I suppose that makes them bad children?).

But it’s a fair point, I wonder how many schemas meaningfully include multiple namespaces?

second, do you really need to intermingle the data from different namespaces? It’s not like this is code. Could you do something like :


{ "localfield1": "foo",
"localfield2": "bar",
"imports": { "namespace1": {"myimportedfield1":"baz"},
"namespace2": {"myotherimportedfield":"bab"}
}
}


Admittedly, I didn’t give this a lot of thought. You’d probably want a more qualified name than “imports”, but seems like this syntax is fairly compact and you get the bonus of pretty simple processing model.

Just a thought.

The infoset != JSON’s data model so any attempt to map the two will either require crippling XML or making JSON as complex as XML. Both choices are unacceptable so I just don’t worry about it.

As for Norm’s article, I’m going to stay away from this issue because I already have more things to deal with then I want to. :)

I guess the question here is whether it’s more important that the naming system is compatible with Java or with XML (and XML applications that rely on XML namespaces, such as Atom or WebDAV). It seems to me that disallowing XML namespace URIs as JSON namespace names may make it hard to use when a service talks both JSON and XML.

For the question about resolvability of names, have a look at Norman’s latest blog entry (at http://norman.walsh.name/2006/07/25/namesAndAddresses).

Best regards, Julian