Turn Freeform MDX Content into Structured Data with Contentlayer

Apr 25
·
views
·
likes

The second part of this series explores setting up and configuring the content layer of a Next.js + MDX blogging system.

Problem

Solution

Contentlayer is a tool that helps deal with the chaos of freeform MDX content by transforming it into structured type-safe data and significantly reducing the boilerplate and external tools required to effectively integrate MDX content with the rest of your app. Folks have been describing it as the "prisma for content".

I have been using and enjoying Contentlayer for my own blog and last week it finally went into beta. Judging by the overwhelmingly positive reception, it seems like I'm not the only one who thinks it's a good idea:

Until today, the only option for JS devs to mesh data all in one place was to use Gatsby + GraphQL. Now you have a lightweight, typesafe "content SDK" to use anywhere🤯 It's been a huge privilege seeing @schickling work on @Contentlayerdev. His instinct for DX is world-class!
If you bring content-as-data into your website, whether it's from Markdown files or a hosted CMS, do yourself a favour and check this out. Massive leap forward in speed, type safety, and DX 👏🏻 (much like @prisma was for databases, since @schickling's working on it, of course!)
This could be one of the next most important developer tools: "Prisma for data" that lets you define and generate typesafe access to any data source. Created by @schickling, founder of Prisma!
Working with content (Markdown files, CMS, ...) is surprisingly difficult when developing modern sites. Contentlayer (now in beta) is a content SDK which aims to make content easy for developers with a focus on great DX & performance.

Code

You're welcome to follow along with this step-by-step walkthrough or if you just want the complete code you can find it at the bottom of this post.

Install and Configure Contentlayer

Install the Contentlayer packages inside your Next.js project:

Terminal
npm install contentlayer next-contentlayer

Stitch Next.js and Contentlayer together by using the next-contentlayer plugin to hook into the next dev and next build processes. This automatically runs the Contentlayer CLI while you're developing locally or building your app for deployment:

next.config.js
const { withContentlayer } = require("next-contentlayer")
 
const nextConfig = {}
 
module.exports = withContentlayer(nextConfig)

Define a Document Type

Create the schema for your blog post document type:

contentlayer.config.js
import { defineDocumentType, makeSource } from "contentlayer/source-files"
 
export const Post = defineDocumentType(() => ({
  name: "Post",
  contentType: "mdx",
}))
 
export default makeSource({
  documentTypes: [Post],
})

And tell contentlayer where your posts source MDX files will live:

contentlayer.config.js
import { defineDocumentType, makeSource } from "contentlayer/source-files"
 
export const Post = defineDocumentType(() => ({
  name: "Post",
  contentType: "mdx",
  // Location of Post source files (relative to `contentDirPath`)
  filePathPattern: "posts/*.mdx",
  // At the time of writing, we also have to define the `fields`
  // option to prevent an error on generation. We'll discuss
  // this option later. For now, we'll add an empty object.
  fields: {},
}))
 
export default makeSource({
  // Location of source files for all defined documentTypes
  contentDirPath: "content",
  documentTypes: [Post],
})

Lastly, create a "hello world" post inside the configured directory for your post document e.g. contentDirPath + filePathPattern = /content/posts

content/posts/hello-world.mdx
# Hello World
 
This is my first post...

Saving this post file while next dev is running will incrementally generate type definitions, helper functions and transformed static files for each document type you have defined in:
makeSource({ documentTypes: [...] })

next dev terminal output
wait - compiling...
File updated: content/posts/hello-world.mdx
Generated 1 document in .contentlayer

Files Generated by Contentlayer

Skip to next section (you don't need to understand this)

Folder structure after generation
.contentlayer
generated
Post
posts__hello-world.mdx.json
index.js
types.d.ts
content
posts
hello-world.mdx
contentlayer.config.js
next.config.js

Contentlayer transforms your posts markdown source file (line 9) into structured data and stores the result as a static json file (line 4). In addition, Contentlayer generates type definitions (line 6) and convenience helper functions for accessing post data (line 5).

The processed json file includes the raw markdown content, the processed MDX code, and generated fields such as the file's name. It will also include custom fields which we will add later:

Truncated .contentlayer/generated/Post/posts__hello-world.mdx.json
{
  "type": "Post",
  "_id": "posts/hello-world.mdx",
  "_raw": {
    "sourceFileName": "hello-world.mdx",
    "contentType": "mdx"
    // [...]
  },
  "body": {
    "raw": "\n# Hello World\n\nThis is my first post....\n",
    "code": "var Component=(()=>{var m=Object.create; [...]"
  }
}

TypeScript and Git Integration

Add Contentlayer to your TypeScript configuration to make it easier to import type definition and helper functions:

tsconfig.json
{
  "compilerOptions": {
    "baseUrl": ".",
    "paths": {
      "contentlayer/generated": ["./.contentlayer/generated"]
    }
  },
  // prettier-ignore
  "include": [
    "next-env.d.ts", 
    "**/*.ts", 
    "**/*.tsx", 
    ".contentlayer/generated",
  ]
}

I'd also recommend adding the .contentlayer folder to your .gitignore to reduce the git commit and diff noise from generated files:

.gitignore
.contentlayer

Single Blog Post Page

Create a "dynamic template" to handle all your single posts by creating a new dynamic route inside the pages/blog/ directory:

.contentlayer [...]
content [...]
pages
blog
[slug].js
[...]
pages/blog/[slug].tsx
export const getStaticPaths = () => {...}
 
export const getStaticProps = () => {...}
 
export default function SinglePostPage() {...}

Inside your dynamic route, import allPosts, an array of all the posts Contentlayer has conveniently processed and generated for you:

pages/blog/[slug].tsx
import { allPosts } from "contentlayer/generated"
 
export const getStaticPaths = () => {...}
 
export const getStaticProps = () => {...}
 
export default function SinglePostPage() {...}

Provide getStaticPaths() a list of posts to pre-render by mapping over allPosts and transforming each posts file name into a slug:

pages/blog/[slug].tsx
import { allPosts } from "contentlayer/generated"
 
export const getStaticPaths = () => {
  return {
    paths: allPosts.map((post) => ({
      params: {
        slug: post._raw.sourceFileName
          // hello-world.mdx => hello-world
          .replace(/\.mdx$/, ""),
      },
    })),
    fallback: false,
  }
}
 
export const getStaticProps = () => {...}
 
export default function SinglePostPage() {...}

Inside getStaticProps() find the current post being generated and pass it down to the SinglePostPage component:

pages/blog/[slug].tsx
import { allPosts } from "contentlayer/generated"
 
export const getStaticPaths = () => {...}
 
export const getStaticProps = ({ params }) => {
  return {
    props: {
      post: allPosts.find(
        (post) =>
          post._raw.sourceFileName.replace(/\.mdx$/, "") === params?.slug,
      ),
    },
  }
}
 
export default function SinglePostPage({ post }) {
  return <div></div>
}
🐣

Next.js will pre-render each post by looping through the list of posts you provided to getStaticPaths(), fetching the processed data from Contentlayer, passing the data down to your page component as props via getStaticProps(), rendering the SinglePostPage() page component and saving the result as static files ready for deployment.

Computed Fields

In addition to Contentlayer's automatically generated fields, you can add your own fields to a document type. One type of custom field is a computed field. Computed fields are useful for adding a new field to a document that uses its current contents to calculate something new. For instance, you could use the contents of a post to calculate an estimated read time.

You may have noticed that the .replace(/\.mdx$/, "") logic is currently being used in both getStaticPaths and getStaticProps. You could move this logic into a single computed field:

contentlayer.config.js
export const Post = defineDocumentType(() => ({
  name: "Post",
  filePathPattern: "posts/*.mdx",
  computedFields: {
    slug: {
      type: "string",
      resolve: (post) =>
        post._raw.sourceFileName
          // hello-world.mdx => hello-world
          .replace(/\.mdx$/, ""),
    },
  },
}))

Saving the config file will trigger a regeneration resulting in the generated json file having a new slug field:

.contentlayer/generated/Post/posts__hello-world.mdx.json
{
  "type": "Post",
  "_id": "posts/hello-world.mdx",
  "slug": "hello-world"
  // [...]
}

slug is now a computed field available to all posts. This makes using allPosts slightly more ergonomic:

pages/blog/[slug].tsx
export const getStaticPaths = () => {
  return {
    paths: allPosts.map((post) => ({
      params: {
-       slug: post._raw.sourceFileName.replace(/\.mdx$/, ""),
+       slug: post.slug,
      },
    })),
    fallback: false,
  }
}
 
export const getStaticProps = ({ params }) => {
  return {
    props: {
-     post: allPosts.find(
-       (post) =>
-         post._raw.sourceFileName.replace(/\.mdx$/, "") === params?.slug,
-     ),
+     post: allPosts.find((post) => post.slug === params?.slug),
    },
  }
}
 
export default function SinglePostPage() {...}

Custom Fields

Similarly to computed fields, you can add custom fields to a document type. These are useful for post metadata such as a title, date, or category.

You can require all posts to have a title field:

contentlayer.config.js
export const Post = defineDocumentType(() => ({
  name: "Post",
  filePathPattern: "posts/*.mdx",
  fields: {
    title: {
      type: "string",
      required: true,
    },
  },
  computedFields: {...},
}))

Type-Safety and Data Integrity

When you save the contentlayer.config.js file, you will notice an error in your terminal:

Warning: Found 1 problem in 1 document.

 └── Missing required fields for 1 document.

     • "content/posts/hello-world.mdx" is missing the following required fields:
       • title: string

Contentlayer will warn you because it has detected that one of your posts is missing the now required title field. This is how Contentlayer adds some type-safety to your freeform content and protects the integrity of your data.

Adding a title to your blog post (in the frontmatter format) will fix the error:

content/posts/hello-world.mdx
+ ---
+ title: "Hello World"
+ ---
 
- # Hello World
+ # A Captivating Title
 
This is my first post...

You can use the type definitions from Contentlayer and Next.js to ensure that the post object passed to your SinglePostPage component is type-safe.

pages/blog/[slug].tsx
import { allPosts, type Post } from "contentlayer/generated"
import { type GetStaticProps, type InferGetStaticPropsType } from "next"
 
export const getStaticPaths = () => {
  return {
    paths: allPosts.map((post) => ({ params: { slug: post.slug } })),
    fallback: false,
  }
}
 
export const getStaticProps: GetStaticProps<{
  post: Post
}> = ({ params }) => {
  const post = allPosts.find((post) => post.slug === params?.slug)
 
  if (!post) {
    return { notFound: true }
  }
 
  return { props: { post } }
}
 
export default function SinglePostPage({
  post,
}: InferGetStaticPropsType<typeof getStaticProps>) {
  return <h1>{post.title}</h1>
}

Blog Post List Page

You can use a similar process to create a list page for all your blog posts:

pages
blog
index.js
[slug].js
pages/blog/index.tsx
import { allPosts, type Post } from "contentlayer/generated"
import { type GetStaticProps, type InferGetStaticPropsType } from "next"
import Link from "next/link"
 
export const getStaticProps: GetStaticProps<{
  posts: Post[]
}> = () => {
  return { props: { posts: allPosts } }
}
 
export default function PostListPage({
  posts,
}: InferGetStaticPropsType<typeof getStaticProps>) {
  return (
    <div>
      <h1>Blog</h1>
 
      {posts.map((post) => (
        <div key={post.slug}>
          <h2>
            <Link href={`/blog/${post.slug}`}>
              <a>{post.title}</a>
            </Link>
          </h2>
        </div>
      ))}
    </div>
  )
}

MDX Components

The ability to use React.js components inside markdown is the main motivation for using MDX. You can use the useMDXComponent() hook to render the current posts "MDX enriched" markdown inside the SinglePostPage component:

pages/blog/[slug].tsx
import { allPosts, type Post } from "contentlayer/generated"
import { type GetStaticProps, type InferGetStaticPropsType } from "next"
import { useMDXComponent } from "next-contentlayer/hooks"
 
export const getStaticPaths = () => {...}
 
export const getStaticProps = () => {...}
 
export default function SinglePostPage({
  post,
}: InferGetStaticPropsType<typeof getStaticProps>) {
  const MDXContent = useMDXComponent(post.body.code)
 
  return (
    <div>
      <h1>{post.title}</h1>
      <MDXContent />
    </div>
  )
}

If we visit /blog/hello-world we will see the post content that we wrote in markdown has been transformed into HTML.

Essentially, we wrote a post in MDX (markdown):

hello-world.mdx
# A Captivating Title
 
This is my first post...

Which Contentlayer (using a bundler under the hood) transformed into JSX and cached as static json:

Truncated unstringified code from contentlayer hello-world.mdx.json
var Component = (() => {
  // ...
  function m(t) {
    let n = Object.assign({ h1: "h1", p: "p" }, t.components)
    return (0, e.jsxs)(e.Fragment, {
      children: [
        (0, e.jsx)(n.h1, { children: "A Captivating Title" }),
        (0, e.jsx)(n.p, { children: "This is my first post..." }),
      ],
    })
  }
  // ...
})()
return Component

Which React rendered and Next.js cached as static HTML:

Truncated static generation payload
<h1>A Captivating Post</h1>
<p>This is my first post...</p>

Closing Thoughts

This covers the majority of setup and configuration. As you've hopefully realized, Contentlayer greatly simplifies the process of implementing MDX.

We didn't have to choose and configure an MDX processor like mdx-bundler, we didn't use any file-system APIs or have to install third-party glob or path packages, we didn't even have to think about ASTs or the complex unified, remark or rehype ecosystem.

On top of that, Contentlayer adds much more to the table like type-safety, convenience functions for dealing with your content, and a single and structured place to define and customize your freeform data.

There is a lot more to MDX components that we will explore in detail in the next post — including how to embed interactive components, re-map markdown constructs to custom React.js components, and style your MDX content.

Complete code

The final completed code of all the steps discussed in this post.

Select a file
contentlayer.config.js
import { defineDocumentType, makeSource } from "contentlayer/source-files"
 
const Post = defineDocumentType(() => ({
  name: "Post",
  contentType: "mdx",
  // Location of Post source files (relative to `contentDirPath`)
  filePathPattern: `posts/*.mdx`,
  fields: {
    title: {
      type: "string",
      required: true,
    },
  },
  computedFields: {
    slug: {
      type: "string",
      resolve: (post) =>
        post._raw.sourceFileName
          // hello-world.mdx => hello-world
          .replace(/\.mdx$/, ""),
    },
  },
}))
 
export default makeSource({
  // Location of source files for all defined documentTypes
  contentDirPath: "content",
  documentTypes: [Post],
})
Series
Build a Developer Blog with Next.js

  • Planned: A Fun and Productive Techstack to Build a Developer Blog
  • Turn Freeform MDX Content into Structured Data with Contentlayer
  • Planned: Add Interactive React Components to Static Markdown Content
  • Planned: Syntax Highlighting, the Easy and Performant Way, with Shiki and Rehype Pretty Code
  • Planned: Simple Post Metrics with useSWR and Prisma
  • (WIP) Automatically Generate Branded Open Graph (OG) Images for Your Blog Posts
  • Planned: Light and Dark Mode with Next Themes