What is Moshi?

Moshi is a fast and powerful JSON parsing library for the JVM and Android, built by the former creators of Google’s Gson to address some of its shortcomings and to have an alternative that was actively maintained.

Unlike Gson, Moshi has excellent Kotlin support and supports both reflection based parsing and a kapt-backed codegen backend that eliminates the runtime performance cost in favor of generating adapters during build time. The kotlin-reflect dependency required for doing reflection-based parsing can add up to 1.8 mB to the final binary, so it’s recommended to use the codegen method if possible.

What is an adapter?

An adapter is Moshi-speak for a class that can convert JSON into an object and an instance of that object into JSON. There are multiple types of JSON adapters supported by Moshi. The first is the one demonstrated in their README which contains two methods annotated @ToJson and @FromJson. The former takes an instance of the object and returns a String, and the latter takes a String and returns an instance of the object. This is the simplest type, and should be used for non-complex types that typically can be represented in simpler forms. Here’s the example Moshi uses, and should be all the introduction you need for this particular type.

The other type is similar to what Moshi generates for its kapt-generated adapters, but leverages the @ToJson/@FromJson annotations. The method signatures here are a bit verbose, and these are the ones we’re going to try to build.

Why write your own adapters?

Good question. Consider this example class:

@JsonClass(generateAdapter = true)
class TextParts(val heading: String, val body: String? = null)

Pretty straightforward. The JsonClass annotation with generateAdapter = true will attempt to use the codegen backend to write an adapter automatically for this. Let’s try converting this to JSON.

val text = TextParts("This is the heading", "And this is the body")
val moshi = Moshi.Builder().build()
// TextPartsJsonAdapter was generated by the codegen backend
println(TextPartsJsonAdapter(moshi).toJson(text))
{"heading":"This is the heading","body":"And this is the body"}

What this means is, given a JSON object that looks like this

{ "heading": "This is the heading", "body": "And this is the body" }

We can get an instance of TextParts that looks like this

val text = TextParts("This is the heading", "And this is the body")

Cool! Now, let’s make things unfortunate. Imagine your backend team is stretched thin, and due to a limitation with how they initially built their database schema, you can only get the above JSON in this form

{
  "heading": "This is the heading",
  "extras": { "body": "And this is the body" }
}

If you try to parse this with the old TextPartsJsonAdapter, your app is going to crash, because the JSON and its Kotlin representation have diverged. The equivalent Kotlin for this new JSON is going to be something like this:

@JsonClass(generateAdapter = true)
class Extras(val body: String? = null)

@JsonClass(generateAdapter = true)
class TextParts(val heading: String, val extras: Extras? = null)

Many things changed here. Your direct access to the body field now needs to go through extras, which just isn’t that nice. You’re also now incurring the (albeit miniscule) overhead of generating two adapters rather than one. Wouldn’t it be great if we could continue to have a flat object like before? Let’s try to make that happen.

How to write your own Moshi adapter?

With less effort than one might think! Let’s put down the basic building blocks.

class TextPartsJsonAdapter {
  // Moshi is flexible about the parameters of these two methods, and for simpler types
  // you will find it easier to follow the example from the Moshi README which does not
  // use JsonReader/JsonWriter and instead directly converts items to and from their String
  // representations. The method names are also not enforced, as Moshi only uses the
  // annotations to find relevant methods. The internal implementation of how they do it
  // can be found here: https://git.io/JLwnb

  @FromJson
  fun fromJson(reader: JsonReader): TextParts? {
    TODO("Not implemented")
  }

  @ToJson
  fun toJson(writer: JsonWriter: value: TextParts?) {
    TODO("Not implemented")
  }
}

Now we’re ready to start parsing. First, let’s implement the toJson part, where we take an instance of the object and then try to write the equivalent JSON for it. Since this is comparatively easier, I’m going to do it in one go and leave comments inline to explain what’s happening.

@ToJson
fun toJson(writer: JsonWriter: value: TextParts?) {
  // Null values shouldn't arrive to the adapter, this error lets callers know
  // what builder options need to be passed to the Moshi.Builder() instance
  // to avoid this particular situation.
  if (value == null) {
    throw NullPointerException("value was null! Wrap in .nullSafe() to write nullable values.")
  }
  // Use the Kotlin `with` scoping method so we don't need to call
  // all methods with the `writer.` prefix.
  with(writer) {
    // Start the JSON object.
    beginObject()

    // Since our `extras` field is nullable, and our backend will send
    // it as a literal null rather than skip it, we want null values to
    // be written into the final JSON.
    serializeNulls = true

    // Create a JSON field with the name 'heading'
    name("heading")

    // Set the value of the 'heading' field to the actual heading
    value(value.heading)

    // Create the 'extras' field
    name("extras")
    if (value.body != null) {
      // If the body text exists, then start a new object and add a
      // body field
      beginObject()
      name("body")
      value(value.bodyText)
      endObject()
    } else {
      // Otherwise we put down a literal null
      nullValue()
    }

    // End the top-level object.
    endObject()
  }
}

Parsing JSON manually is relatively easy to screw up and Moshi will let you know if you get nesting wrong (missed a closing endObject() or endArray()) and other easily detectable problems, but you should definitely have tests for all possible cases. I’ll let the readers do that on their own, but if you really need to see an example then let me know below.

Anyways, that’s the object -> JSON part sorted. Now let’s try to do the reverse. Here’s where we are as of now.

fun fromJson(reader: JsonReader): TextParts? {
  TODO("Not implemented")
}

Same as writing JSON, we need to start by making an object.

 fun fromJson(reader: JsonReader): TextParts? {
+  // We'll be constructing the object at the end so these
+  // will store the values we read.
+  var heading: String? = null
+  var body: String? = null
+  with(reader) {
+    beginObject()
+    endObject()
+  }
   TODO("Not implemented")
 }

We have a fixed set of keys that we expect to read, so go ahead and configure a couple instances of JsonReader.Options that we will use to find the keys in this JSON.

+val topLevelKeys = JsonReader.Options.of("heading", "extras")
+val extrasKeys = JsonReader.Options.of("body")
+
 fun fromJson(reader: JsonReader): TextParts? {
   // We'll be constructing the object at the end so these
   // will store the values we read.

And we’re set. You’ll see the significance of the Options objects now.

   var body: String? = null
   with(reader) {
     beginObject()
+    while(hasNext()) {
+      when(selectName(topLevelKeys)) {
+        0 -> heading = nextString() ?: throw Util.unexpectedNull(
+          "heading",
+          "text",
+          this
+        )
+      }
+    }
     endObject()
   }
   TODO("Not implemented")

reader.hasNext() is going to continue iterating through the document’s tokens until it’s completed, which lets us look through the entire document for the parts we need. The selectName(JsonReader.Options) method will return the index of a matched key, so 0 there means that the heading key was found. In response to that, we want to read it as a string and throw if it is null (since it’s non-nullable in TextParts). The Util.unexpectedNull method is a little nicety that is part of Moshi’s internals and is used by its kapt-generated adapters to provide better error messages and we’re going to do the same.

         0 -> heading = nextString() ?: throw Util.unexpectedNull(
           "heading",
           "text",
           this
         )
+        -1 -> {
+          // Skip unknown values
+          skipName()
+          skipValue()
+        }
       }
     }
     endObject()

When I said that selectName returns the index of the matched key, I didn’t mention that it returns -1 when it comes across a key that isn’t in the Options object. Since we don’t care about them, we’re going to skip both their name and value and continue right on ahead. Now, we’re going to try and parse that inner extras object. A lot is about to happen quickly, but bear with me as I explain things.

           "text",
           this
         )
+        1 -> {
+          // "extras" is nullable, so we first try to see if it is null.
+          // If it isn't, this will throw and we can then safely assume
+          // a non-null value and proceed.
+          try {
+            nextNull<Any>()
+          } catch (_: JsonDataException) {
+            beginObject()
+            while (hasNext()) {
+              when (selectName(extrasKeys)) {
+                0 -> body = nextString()
+                -1 -> {
+                  // Skip unknown values
+                  skipName()
+                  skipValue()
+                }
+              }
+            }
+            endObject()
+          }
+        }
         -1 -> {
           // Skip unknown values
           skipName()
           skipValue()

Now that you look at it, not really that different from what we did above. The only new thing here is the nextNull method, which simply tries to find a null value and throws the JsonDataException if the value wasn’t null.

     }
     endObject()
   }
-  TODO("Not implemented")
+  // Satisfy the typechecker and throw in case the JSON body
+  // didn't contain the 'heading' field at all
+  require(heading != null) { "heading must not be null" }
+  return TextParts(heading, body)
 }

And that’s it! The final adapter is going to look like this

class TextPartsJsonAdapter {
  val topLevelKeys = JsonReader.Options.of("heading", "extras")
  val extrasKeys = JsonReader.Options.of("body")

  @FromJson
  fun fromJson(reader: JsonReader): TextParts? {
    // We'll be constructing the object at the end so these
    // will store the values we read.
    var heading: String? = null
    var body: String? = null
    with(reader) {
      beginObject()
      while(hasNext()) {
        when(selectName(topLevelKeys)) {
          0 -> heading = nextString() ?: throw Util.unexpectedNull(
            "heading",
            "text",
            this
          )
          1 -> {
            // "extras" is nullable, so we first try to see if it is null.
            // If it isn't, this will throw and we can then safely assume
            // a non-null value and proceed.
            try {
              nextNull<Any>()
            } catch (_: JsonDataException) {
              beginObject()
              while (hasNext()) {
                when (selectName(extrasKeys)) {
                  0 -> body = nextString()
                  else -> {
                    // Skip unknown
                    skipName()
                    skipValue()
                  }
                }
              }
              endObject()
            }
          }
          -1 -> {
            skipName()
            skipValue()
          }
        }
      }
      endObject()
    }
    // Satisfy the typechecker and throw in case the JSON body
    // didn't contain the 'heading' field at all
    require(heading != null) { "heading must not be null" }
    return TextParts(heading, body)
  }

  @ToJson
  fun toJson(writer: JsonWriter: value: TextParts?) {
    // Null values shouldn't arrive to the adapter, this error lets callers know
    // what builder options need to be passed to the Moshi.Builder() instance
    // to avoid this particular situation.
    if (value == null) {
      throw NullPointerException("value was null! Wrap in .nullSafe() to write nullable values.")
    }
    // Use the Kotlin `with` scoping method so we don't need to call
    // all methods with the `writer.` prefix.
    with(writer) {
      // Start the JSON object.
      beginObject()

      // Since our `extras` field is nullable, and our backend will send
      // it as a literal null rather than skip it, we want null values to
      // be written into the final JSON.
      serializeNulls = true

      // Create a JSON field with the name 'heading'
      name("heading")

      // Set the value of the 'heading' field to the actual heading
      value(value.heading)

      // Create the 'extras' field
      name("extras")
      if (value.body != null) {
        // If the body text exists, then start a new object and add a body field
        beginObject()
        name("body")
        value(value.bodyText)
        endObject()
      } else {
        // Otherwise we put down a literal null
        nullValue()
      }

      // End the top-level object.
      endObject()
    }
  }
}

This is certainly a lengthy job to do, and this blog post is a result of nearly 8 hours I spent writing JSON adapters by hand. Certainly not recommended if avoidable, but sometimes you just need to. When it comes to it, now you hopefully know how :)