guisehn.com

Converting complex JSON into Elixir structs

Converting complex or deeply nested JSON into Elixir structs can be tedious if you write all the conversion logic manually. But did you know Ecto can handle this seamlessly — outside a database context, purely in memory?

For this example, we’ll convert the JSON:

{
  "id": 1,
  "name": "Alice Johnson",
  "email": "alice@example.com",
  "profile": {
    "age": 30,
    "location": "New York",
    "preferences": {
      "theme": "dark",
      "notifications": true
    }
  },
  "addresses": [
    {
      "type": "home",
      "street": "123 Main St",
      "city": "New York",
      "state": "NY",
      "zip": "10001"
    },
    {
      "type": "work",
      "street": "456 Business Rd",
      "city": "New York",
      "state": "NY",
      "zip": "10005"
    }
  ]
}

so that it becomes a nested struct:

%MyApp.User{
  id: 1,
  name: "Alice Johnson",
  email: "alice@example.com",
  profile: %MyApp.User.Profile{
    age: 30,
    location: "New York",
    preferences: %MyApp.User.Profile.Preferences{
      theme: "dark",
      notifications: true
    }
  },
  addresses: [
    %MyApp.User.Address{
      type: "home",
      street: "123 Main St",
      city: "New York",
      state: "NY",
      zip: "10001"
    },
    %MyApp.User.Address{
      type: "work",
      street: "456 Business Rd",
      city: "New York",
      state: "NY",
      zip: "10005"
    }
  ]
}

Ecto embedded schema to the rescue!

Ecto.Schema provides embedded_schema, embeds_one, and embeds_many, which allow us to define in-memory schemas.

We can use them to define our structure, and it will automatically create the nested structs:

defmodule MyApp.User do
  use Ecto.Schema

  @primary_key false

  embedded_schema do
    field :id, :integer
    field :name, :string
    field :email, :string

    embeds_one :profile, Profile, primary_key: false do
      field :age, :integer
      field :location, :string

      embeds_one :preferences, Preferences, primary_key: false do
        field :theme, :string
        field :notifications, :boolean
      end
    end

    embeds_many :addresses, Address, primary_key: false do
      field :type, :string
      field :street, :string
      field :city, :string
      field :state, :string
      field :zip, :string
    end
  end
end

Notice primary_key: false on each level: without that, Ecto would automatically add an integer id field to each struct.

Parsing JSON to the Ecto schema

This part is slightly tricky, but we can simplify it with just two small functions!

To instantiate an Ecto schema from an arbitrary JSON/map, Ecto has the functions cast/3 + apply_changes/1 in Ecto.Changeset. The tricky part is that cast requires us to pass the list of keys we want to allow, e.g.:

iex(1)> data = %{"id" => 1, "name" => "Alice Johnson", "email" => "alice@example.com", ...}
%{"id" => 1, "name" => "Alice Johnson", ...}

#                                                 ↓ we need to pass all fields here
iex(2)> Ecto.Changeset.cast(%MyApp.User{}, data, [:id, :name])
%MyApp.User{id: 1, name: "Alice Johnson", email: nil}

Ecto likely enforces an allowlist of keys because cast/3 is primarily used in web request contexts, ensuring users can’t modify protected database columns. But in our case, we just want to convert the JSON map to the struct allowing all keys. Manually writing them, especially with a nested object structure, would be cumbersome.

Fortunately, Ecto provides reflection utilities, which we can use to auto-generate the keys for allow listing. So let’s do it:

defmodule MyApp.User do
  # [all the previous code here]

  def from_json(data) do
    %MyApp.User{}
    |> build_changeset(data)
    |> Ecto.Changeset.apply_changes()
  end

  defp build_changeset(schema, data) do
    struct = schema.__struct__

    # Gets the field names, e.g. [:id, :name, :email] so that we can
    # pass them to `cast/3`
    fields = struct.__schema__(:fields) - struct.__schema__(:embeds)

    schema
    |> Ecto.Changeset.cast(data, fields)
    |> then(fn schema ->
      # Now we get the nested structures, e.g. `[:profile, :addresses]`
      embeds = struct.__schema__(:embeds)

      Enum.reduce(embeds, schema, fn embed_name, schema ->
        # We call `build_changeset/2` recursively, so that this also runs
        # for the nested structures like `profile`, `addresses`, `preferences`, etc.
        Changeset.cast_embed(schema, embed_name, with: &build_changeset/2)
      end)
    end)
  end
end

And that’s it! Now, we can call:

MyApp.User.from_json(%{"id" => 1, "name" => "Alice Johnson", ...})

and it will automatically parse the nested structure, even taking care of enforcing and converting types where needed.