ScalaPB and JSON

ScalaPB can convert protocol buffers to and from JSON, using json4s.

Setting up your project#

Make sure that you are using ScalaPB 0.5.x or later.

In build.sbt add a dependency on scalapb-json4s:

// For ScalaPB 0.11.x (json4s 0.4.x, released for Scala 3):
libraryDependencies += "com.thesamet.scalapb" %% "scalapb-json4s" % "0.12.0"

// For ScalaPB 0.11.x (json4s 0.3.x):
libraryDependencies += "com.thesamet.scalapb" %% "scalapb-json4s" % "0.11.1"

// For ScalaPB 0.10.x:
libraryDependencies += "com.thesamet.scalapb" %% "scalapb-json4s" % "0.10.1"

// For ScalaPB 0.9.x:
libraryDependencies += "com.thesamet.scalapb" %% "scalapb-json4s" % "0.9.3"

// For ScalaPB 0.8.x:
libraryDependencies += "com.thesamet.scalapb" %% "scalapb-json4s" % "0.7.2"

// For ScalaPB 0.7.x:
libraryDependencies += "com.thesamet.scalapb" %% "scalapb-json4s" % "0.7.2"

// For ScalaPB 0.6.x (note the different groupId):
libraryDependencies += "com.trueaccord.scalapb" %% "scalapb-json4s" % "0.3.2"

// For ScalaPB 0.5.x (note the different groupId):
libraryDependencies += "com.trueaccord.scalapb" %% "scalapb-json4s" % "0.1.6"

In your code, you can now convert to JSON:

import scalapb.json4s.JsonFormat

val r: String = JsonFormat.toJsonString(myProto)

Parse JSON back to a protocol buffer:

import scalapb.json4s.JsonFormat

val proto: MyProto = JsonFormat.fromJsonString[MyProto](
    """{"x": "17"}""")

There are lower-level functions toJson() and fromJson() that convert from protos to json4s's JValue:

def toJson(m: GeneratedMessage): JObject

def fromJson[Proto](value: JValue): Proto

Finally, in JsonFormat there are two implicit methods that instantiate Reader[Proto] and Writer[Proto].

More printing and parsing options#

There are a few more options available to customize the format used to print and parse JSON. To take advantage of that, instantiate Printer and Parser and call toJson() / fromJson() as usual.

For example:

new scalapb.json4s.Printer(
  includingDefaultValueFields = true,
  preservingProtoFieldNames = true,
  formattingLongAsNumber = true
).toJson(myProto)

Options:

includingDefaultValueFields (default: false): should fields that are set to their default value be included in the output.
preservingProtoFieldNames (default: false): by default, field names are mapped to lowerCamelCase and become JSON object keys. Setting this option to true would make the parser and the printer use the original field names as specified in the proto file (normally, in snake_case)
formatLongAsNumber (default: false): by default, longs are serialized as strings. To use the numeric representation, set this option to true. Note that due to the way Javascript represents numbers, there is a possibility to lose precision (more details here).

The parser can be instantiated with new scalapb.json4s.Parser(), and various methods can return instances of the parser with customized configuration:

ignoringUnkownFields: by default the parser will throw a JsonFormatException when encountering unknown fields. By enabling this option, unknown options will be silently ignored.
ignoringOverlappingOneofFields: by default the parser will throw a JsonFormatException if values are provided for more than one field within the same oneof. By enabling this option, when more than one field is present for a oneof, one of the values of this field will be picked for the oneof.
mapEntriesAsKeyValuePairs: by default, protobuf maps are modeled as json objects. When this setting is enabled, protobuf maps are expected to be read as arrays of objects with key and value keys.

See the list of constructor paramerters here

Printing and parsing Anys#

In Protocol Buffers, google.protobuf.Any is a type that embeds an arbitrary protobuf message. An Any is represented as a message that contains a typeUrl field that identifies the type, and a bytes field value which contains the serialized contents of a message. In JSON, the message embedded in the Any is serialized as usual, and there is a @type key added to it to identify which message it is. The parser expects this @type key to know which message it is. To accomplish this, all the expected embedded types need to be registered with a TypeRegistry so the printer and parser know how to process the embedded message.

The following example is based on this proto.

import com.thesamet.docs.json._
import scalapb.json4s.{Printer, Parser, TypeRegistry}

val c = MyContainer(
  myAny=Some(
    com.google.protobuf.any.Any.pack(
      MyMessage(x=17)
    )
  )
)
// c: MyContainer = MyContainer(
//   Some(
//     Any(
//       "type.googleapis.com/com.thesamet.docs.MyMessage",
//       <ByteString@76f54881 size=2 contents="\b\021">,
//       UnknownFieldSet(Map())
//     )
//   ),
//   UnknownFieldSet(Map())
// )

val typeRegistry = TypeRegistry().addMessage[MyMessage]
// typeRegistry: TypeRegistry = TypeRegistry(
//   Map(
//     "type.googleapis.com/com.thesamet.docs.MyMessage" -> com.thesamet.docs.json.MyMessage$@717dee2c
//   ),
//   Set()
// )

val printer = new Printer().withTypeRegistry(typeRegistry)
// printer: Printer = scalapb.json4s.Printer@2f3c5217

printer.print(c)
// res0: String = "{\"myAny\":{\"@type\":\"type.googleapis.com/com.thesamet.docs.MyMessage\",\"x\":17}}"

Conversely, you can start from a JSON and parse it back to a MyContainer that contains an Any field:

val parser = new Parser().withTypeRegistry(typeRegistry)
// parser: Parser = scalapb.json4s.Parser@5ca13b8f

parser.fromJsonString[MyContainer]("""
  {
    "myAny": {
      "@type": "type.googleapis.com/com.thesamet.docs.MyMessage",
      "x": 17
    }
  }""")
// res1: MyContainer = MyContainer(
//   Some(
//     Any(
//       "type.googleapis.com/com.thesamet.docs.MyMessage",
//       <ByteString@98fda95 size=2 contents="\b\021">,
//       UnknownFieldSet(Map())
//     )
//   ),
//   UnknownFieldSet(Map())
// )