[golang-dev] Proposal: Allow json.Marshal / json.Unmarshal handle non-string map keys if they are encoding.TextMarshaler / encoding.TextUnmarshaler

[a***@gmail.com]

Currently, json.Marshal will fail to marshal maps with non-string keys,
e.g.:

// http://play.golang.org/p/2m9wLZATqw
type Coord struct { X,Y int }
occupied := map[Coord]bool{}
occupied[Coord{1,2}] = true
data, err := json.Marshal(occupied)
fmt.Printf("Data: %s\nErr: %v", data, err)

I propose to enhance the encoding/json package such that:
(1) for json.Marshal: If the map key is a string kind it is used
directly. Otherwise if the map key satisfies the encoding.TextMarshaler
interface then that is used to generate the map key. Otherwise it fails as
it does today.
(2) for json.Unmarshal: If the map key is a string kind it is written
directly. Otherwise if the map key satisfies the encoding.TextUnmarshaler
interface then that is used to decode the map key. Otherwise it fails as
it does today.

This would, for example, allow http://play.golang.org/p/350eQmK6KL to work.

This would be for the 1.6 release, this discussion is for planning purposes
only.

Thoughts?
- Augusto

[David Symonds]

encoding/json produces JSON. JSON does not permit non-string keys in objects.

[a***@gmail.com]

Sorry, I meant non-string keys on the Go map side. The produced & decoded
JSON must always have string keys for the JSON objects, hence the
requirement for TextMarshaler / TextUnmarshaler rather than json.Marshaler
/ json.Unmarshaler.

Post by David Symonds
encoding/json produces JSON. JSON does not permit non-string keys in objects.

[Russ Cox]

It sounds reasonable to me.

[Chris Hines]

What would the behavior be if to unique value return the same string from MarshalText?

Chris

[Russ Cox]

Post by Chris Hines
What would the behavior be if to unique value return the same string from MarshalText?

Then they're considered the same value by anything that deals with the
marshaled form. (If that hurts, don't do it.)

Russ

[a***@gmail.com]

Indeed, the JSON spec allows multiple identical keys for objects, so as
Russ said it would simply output the repeated key.

Similarly, when decoding a map, it would unmarshal each key and value and
then assign that into the map, so last-write-wins, as is currently
<http://play.golang.org/p/dXcQPNmSF-> done.

An interesting side effect is that it would then be possible to explicitly
handle JSON with repeated keys (or even explicitly record the order of the
json keys) by having the go-side map key TextUnmarshaler have
non-deterministic unmarshaling (e.g. record timestamp for each unmarshal).

- Augusto

Post by Russ Cox

Post by Chris Hines
What would the behavior be if to unique value return the same string from MarshalText?

Then they're considered the same value by anything that deals with the
marshaled form. (If that hurts, don't do it.)
Russ

[Chris Hines]

Very good. I was not aware that JSON allowed duplicate keys.

Chris

Post by a***@gmail.com
Indeed, the JSON spec allows multiple identical keys for objects, so as
Russ said it would simply output the repeated key.
Similarly, when decoding a map, it would unmarshal each key and value and
then assign that into the map, so last-write-wins, as is currently
<http://play.golang.org/p/dXcQPNmSF-> done.
An interesting side effect is that it would then be possible to explicitly
handle JSON with repeated keys (or even explicitly record the order of the
json keys) by having the go-side map key TextUnmarshaler have
non-deterministic unmarshaling (e.g. record timestamp for each unmarshal).
- Augusto

Post by Russ Cox

Post by Chris Hines
What would the behavior be if to unique value return the same string from MarshalText?

Then they're considered the same value by anything that deals with the
marshaled form. (If that hurts, don't do it.)
Russ

[m***@gmail.com]

I gave it a try. First hacky version (encode should be fine, but decode is
ugly).
https://go-review.googlesource.com/#/c/14551/

But I guess also encode should be done differently, in order to avoid to
much slow down.

[Kevin Gillette]

The proposed "if string kind, it is used directly" behavior is the opposite
of the json package's semantics; I suggest we reverse that so that it is:
(1) for json.Marshal: if the map key implements json.Marshaler or
encoding.TextMarshaler, they are used in that order of preference;
otherwise, if the key type is of string kind, the map values are used
directly.
(2) for json.Unmarshal: if the map key implements json.Unmarshaler or
encoding.TextUnmarshaler, they are used in that order of preference;
otherwise, if the key type is of string kind, the JSON object keys are
stored directly.

Post by a***@gmail.com
Currently, json.Marshal will fail to marshal maps with non-string keys,
// http://play.golang.org/p/2m9wLZATqw
type Coord struct { X,Y int }
occupied := map[Coord]bool{}
occupied[Coord{1,2}] = true
data, err := json.Marshal(occupied)
fmt.Printf("Data: %s\nErr: %v", data, err)
(1) for json.Marshal: If the map key is a string kind it is used
directly. Otherwise if the map key satisfies the encoding.TextMarshaler
interface then that is used to generate the map key. Otherwise it fails as
it does today.
(2) for json.Unmarshal: If the map key is a string kind it is written
directly. Otherwise if the map key satisfies the encoding.TextUnmarshaler
interface then that is used to decode the map key. Otherwise it fails as
it does today.
This would, for example, allow http://play.golang.org/p/350eQmK6KL to work.
This would be for the 1.6 release, this discussion is for planning
purposes only.
Thoughts?
- Augusto

[a***@gmail.com]

fyi, the proposal issue is filed
at https://github.com/golang/go/issues/12146

(1) json.Marshaler / json.Unmarshaler is not going to be used. JSON does
not allow structured keys, only string keys.
(2) using encoding.TextMarshaler before string kind is more likely to break
existing programs.

Ideally I agree: encoding.Text(Un)Marshaler should take precedence over the
string kind. However, practically, I'm not sure whether it's worth the
compatibility inconsistency.

- Augusto

Post by Kevin Gillette
The proposed "if string kind, it is used directly" behavior is the
opposite of the json package's semantics; I suggest we reverse that so that
(1) for json.Marshal: if the map key implements json.Marshaler or
encoding.TextMarshaler, they are used in that order of preference;
otherwise, if the key type is of string kind, the map values are used
directly.
(2) for json.Unmarshal: if the map key implements json.Unmarshaler or
encoding.TextUnmarshaler, they are used in that order of preference;
otherwise, if the key type is of string kind, the JSON object keys are
stored directly.

Post by a***@gmail.com
Currently, json.Marshal will fail to marshal maps with non-string keys,
// http://play.golang.org/p/2m9wLZATqw
type Coord struct { X,Y int }
occupied := map[Coord]bool{}
occupied[Coord{1,2}] = true
data, err := json.Marshal(occupied)
fmt.Printf("Data: %s\nErr: %v", data, err)
(1) for json.Marshal: If the map key is a string kind it is used
directly. Otherwise if the map key satisfies the encoding.TextMarshaler
interface then that is used to generate the map key. Otherwise it fails as
it does today.
(2) for json.Unmarshal: If the map key is a string kind it is written
directly. Otherwise if the map key satisfies the encoding.TextUnmarshaler
interface then that is used to decode the map key. Otherwise it fails as
it does today.
This would, for example, allow http://play.golang.org/p/350eQmK6KL to work.
This would be for the 1.6 release, this discussion is for planning
purposes only.
Thoughts?
- Augusto

[Marvin Stenger]

If you still would like to have your own MarshalText method called on your
strings, you have to wrap them and implementing the needed interfaces.