Turn Freeform MDX Content into Structured Data with Contentlayer

Apr, 22
Β·
β€’β€’β€’ views
Β·
β€’β€’β€’ likes
Series
Build a Developer Blog with Next.js

  • A Fun and Productive Techstack to Build a Developer Blog
  • Turn Freeform MDX Content into Structured Data with Contentlayer
  • Add Interactivity and Flare to Markdown with MDX Components
  • Syntax Highlighting, the Easy and Performant Way, with Shiki and Rehype Pretty Code
  • Simple Post Metrics with useSWR and Prisma
  • Light and Dark Mode with Next Themes
  • Automated Open Graph Images with TBA
  • More Parts TBA

The second part of this series explores setting up and configuring the content layer of a Next.js + MDX blogging system.

The Problem

  • Most current MDX solutions leave integrating transformed data inside page templates up to the developer. This creates repetitive boilerplate that relies on file-system APIs and additional third party packages like glob, path and gray-matter.
  • The freeform nature of markdown files makes it difficult to maintain and guarantee structure as the number of posts grow.
  • The only way to extend the MDX processing is through the unified, rehype and remark ecosystem which is powerful but complicated.
  • The combined complexity of related options and configuration scattered across a project is a pain to manage.

The Solution

Contentlayer is a tool that helps deal with the chaos of freeform MDX content by transforming it into structured type-safe data and significantly reducing the boilerplate and external tools required to effectively integrate MDX content with the rest of your app. Folks have been describing it as the "prisma for content".

I have been using and enjoying Contentlayer for my own blog and last week it finally went into beta. Judging by the overwhelmingly positive reception, it seems like I'm not the only one who thinks it's a good idea:

Until today, the only option for JS devs to mesh data all in one place was to use Gatsby + GraphQL. Now you have a lightweight, typesafe "content SDK" to use anywhere🀯 It's been a huge privilege seeing @schickling work on @Contentlayerdev. His instinct for DX is world-class!
If you bring content-as-data into your website, whether it's from Markdown files or a hosted CMS, do yourself a favour and check this out. Massive leap forward in speed, type safety, and DX πŸ‘πŸ» (much like @prisma was for databases, since @schickling's working on it, of course!)
This could be one of the next most important developer tools: "Prisma for data" that lets you define and generate typesafe access to any data source. Created by @schickling, founder of Prisma!

If you'd like to know more about why Contentlayer exists and what problems it elegantly solves, I'd recommend this thread:

Working with content (Markdown files, CMS, ...) is surprisingly difficult when developing modern sites. Contentlayer (now in beta) is a content SDK which aims to make content easy for developers with a focus on great DX & performance.

Or if you really want to dig deep, this longform interview between Lee Robinson and the creator of Contentlayer.

Install and Configure Contentlayer

Install the Contentlayer packages inside your Next.js project:

Terminal
npm install contentlayer next-contentlayer

Stitch Next.js and Contentlayer together by using the next-contentlayer plugin to hook into the next dev and next build processes. This automatically runs the Contentlayer CLI while you're developing locally or building your app for deployment:

next.config.js
import { withContentlayer } from "next-contentlayer"
 
export default withContentlayer(
  {}, // Next.js config here
)

Define a Document Type

Create the schema for your blog post document type:

contentlayer.config.js
import { defineDocumentType, makeSource } from "contentlayer/source-files"
 
export const Post = defineDocumentType(() => ({
  name: "Post",
}))
 
export default makeSource({
  documentTypes: [Post],
})

And tell contentlayer where your posts source MDX files will live:

contentlayer.config.js
import { defineDocumentType, makeSource } from "contentlayer/source-files"
 
export const Post = defineDocumentType(() => ({
  name: "Post",
  // Location of Post source files (relative to `contentDirPath`)
  filePathPattern: "posts/*.mdx",
}))
 
export default makeSource({
  // Location of source files for all defined documentTypes
  contentDirPath: "data",
  documentTypes: [Post],
})

Lastly, create a "hello world" post inside the configured directory for your post document e.g. contentDirPath + filePathPattern = /data/posts

data/posts/hello-world.mdx
# Hello World
 
This is my first post...

Saving this post file while next dev is running will incrementally generate type definitions, helper functions and transformed static files for each document type you have defined in:
makeSource({ documentTypes: [...] })

next dev terminal output
wait - compiling...
File updated: data/posts/hello-world.mdx
Generated 1 document in .contentlayer

Files Generated by Contentlayer

Skip to next section (you don't need to understand this)

Folder structure after generation
.contentlayer
generated
Post
posts__hello-world.mdx.json
index.js
types.d.ts
data
posts
hello-world.mdx
contentlayer.config.js
next.config.js

Contentlayer transforms your posts markdown source file (line 10) into structured data and stores the result as a static json file (line 5). In addition, Contentlayer generates type definitions (line 7) and convenience helper functions for accessing post data (line 6).

The processed json file includes the raw markdown content, the processed MDX code and generated fields such as the file's name. It will also include custom fields we will add later:

Truncated .contentlayer/generated/Post/posts__hello-world.mdx.json
{
  "type": "Post",
  "_id": "posts/hello-world.mdx",
  "_raw": {
    "sourceFileName": "hello-world.mdx",
    "contentType": "mdx"
    // [...]
  },
  "body": {
    "raw": "\n# Hello World\n\nThis is my first post....\n",
    "code": "var Component=(()=>{var m=Object.create; [...]"
  }
}

TypeScript and Git Integration

Add Contentlayer to your TypeScript configuration to make it easier to import type definition and helper functions:

tsconfig.json
{
  "compilerOptions": {
    "baseUrl": ".",
    "paths": {
      "contentlayer/generated": ["./.contentlayer/generated"]
    }
  },
  // prettier-ignore
  "include": [
    "next-env.d.ts", 
    "**/*.ts", 
    "**/*.tsx", 
    ".contentlayer/generated",
  ]
}

I'd also recommend adding the .contentlayer folder to your .gitignore to reduce the git commit and diff noise from generated files:

.gitignore
.contentlayer

Single Blog Post Page

Create a "dynamic template" to handle all your single posts by creating a new dynamic route inside the pages/blog/ directory:

.contentlayer [...]
data [...]
pages
blog
[slug].js
[...]
pages/blog/[slug].tsx
export const getStaticPaths = () => {...}
 
export const getStaticProps = () => {...}
 
export default function SinglePostPage() {...}

Inside your dynamic route, import allPosts, an array of all the posts Contentlayer has conveniently processed and generated for you:

pages/blog/[slug].tsx
import { allPosts } from "contentlayer/generated"
 
export const getStaticPaths = () => {...}
 
export const getStaticProps = () => {...}
 
export default function SinglePostPage() {...}

Provide getStaticPaths() a list of posts to pre-render by mapping over allPosts and transforming each posts file name into a slug:

pages/blog/[slug].tsx
import { allPosts } from "contentlayer/generated"
 
export const getStaticPaths = () => {
  return {
    paths: allPosts.map((post) => ({
      slug: post._raw.sourceFileName
        // hello-world.mdx => hello-world
        .replace(/\.mdx$/, ""),
    })),
    fallback: false,
  }
}
 
export const getStaticProps = () => {...}
 
export default function SinglePostPage() {...}

Inside getStaticProps() find the current post being generated and pass it down to the SinglePostPage component:

pages/blog/[slug].tsx
import { allPosts } from "contentlayer/generated"
 
export const getStaticPaths = () => {...}
 
export const getStaticProps = ({ params }) => {
  return {
    props: {
      post: allPosts.find(
        (post) =>
          post._raw.sourceFileName.replace(/\.mdx$/, "") === params.slug,
      ),
    },
  }
}
 
export default function SinglePostPage({ post }) {
  return <div></div>
}

🐣
Next.js will pre-render each post by looping through the list of posts you provided to getStaticPaths(), fetching the processed data from Contentlayer, passing the data down to your page component as props via getStaticProps(), rendering the SinglePostPage() page component and saving the result as static files ready for deployment.

Computed Fields

In addition to Contentlayers automatically generated fields, you can add your own fields to a document type. One type of custom field is a computed field. Computed fields are useful for adding a new field to a document that uses its current contents to calculate something new. For instance you could use a posts contents to calculate an estimated read time.

You may have noticed that the .replace(/\.mdx$/, "") logic is currently being used in both getStaticPaths and getStaticProps. You could move this logic into a single computed field:

contentlayer.config.js
export const Post = defineDocumentType(() => ({
  name: "Post",
  filePathPattern: "posts/*.mdx",
  computedFields: {
    slug: {
      type: "string",
      resolve: (post) =>
        post._raw.sourceFileName
          // hello-world.mdx => hello-world
          .replace(/\.mdx$/, ""),
    },
  },
}))

Saving the config file will trigger a regeneration resulting in the generated json file having a new slug field:

.contentlayer/generated/Post/posts__hello-world.mdx.json
{
  "type": "Post",
  "_id": "posts/hello-world.mdx",
  "slug": "hello-world"
  // [...]
}

slug is now a computed field available to all posts. This makes using allPosts slightly more ergonomic:

pages/blog/[slug].tsx
export const getStaticPaths = () => {
  return {
    paths: allPosts.map((post) => ({
-     slug: post._raw.sourceFileName.replace(/\.mdx$/, ""),
+     slug: post.slug,
    })),
    fallback: false,
  }
}
 
export const getStaticProps = ({ params }) => {
  return {
    props: {
-     post: allPosts.find(
-       (post) =>
-         post._raw.sourceFileName.replace(/\.mdx$/, "") === params.slug,
-     ),
+     post: allPosts.find((post) => post.slug === params.slug),
    },
  }
}
 
export default function SinglePostPage() {...}

Custom Fields

Similarly to computed fields, you can add custom fields to a document type. These are useful for post metadata such as a title, date or category.

You can require all posts to have a title field:

contentlayer.config.js
export const Post = defineDocumentType(() => ({
  name: "Post",
  filePathPattern: "posts/*.mdx",
  fields: {
    title: {
      type: "string",
      required: true,
    },
  },
  computedFields: {...},
}))

Type-Safety and Data Integrity

When you save the contentlayer.config.js file, you will notice an error in your terminal:

Warning: Found 1 problem in 1 document.

 └── Missing required fields for 1 document.

     β€’ "data/posts/hello-world.mdx" is missing the following required fields:
       β€’ title: string

Contentlayer will warn you because it has detected that one of your posts is missing the now required title field. This is how Contentlayer adds some type-safety to your freeform content and protects the integrity of your data.

Adding a title to your blog post (in the frontmatter format) will fix the error:

data/posts/hello-world.mdx
---
title: "Hello World"
---
 
# Hello World
 
This is my first post...

You can use the type definitions from Contentlayer and Next.js to ensure that the post object passed to your SinglePostPage component is type-safe.

pages/blog/[slug].tsx
import { allPosts } from "contentlayer/generated"
import type { GetStaticProps, InferGetStaticPropsType } from "next"
import type { Post } from "contentlayer/generated"
 
export const getStaticPaths = () => {
  return {
    paths: allPosts.map((post) => ({ slug: post.slug })),
    fallback: false,
  }
}
 
export const getStaticProps: GetStaticProps<{
  post: Blog
}> = ({ params }) => {
  const post = allPosts.find((post) => post.slug === params.slug)
 
  if (!post) {
    return { notFound: true }
  }
 
  return { props: { post } }
}
 
export default function SinglePostPage({
  post,
}: InferGetStaticPropsType<typeof getStaticProps>) {
  return <h1>{post.title}</h1>
}

Blog Post List Page

You can use a similar process to create a list page for all your blog posts:

pages
blog
index.js
[slug].js
pages/blog/index.tsx
import Link from "next/link"
import { allPosts } from "contentlayer/generated"
 
export const getStaticProps: GetStaticProps<{
  posts: Blog[]
}> = () => {
  return { props: { posts: allPosts } }
}
 
export default function PostListPage({
  posts,
}: InferGetStaticPropsType<typeof getStaticProps>) {
  return (
    <div>
      <h1>Blog</h1>
 
      {posts.map((post) => (
        <div key={post.slug}>
          <h2>
            <Link href={`/blog/${post.slug}`}>
              <a>{post.title}</a>
            </Link>
          </h2>
        </div>
      ))}
    </div>
  )
}

MDX Components

The ability to use React.js components inside markdown is the main motivation for using MDX. You can use the useMDXComponent() hook to render the current posts "MDX enriched" markdown inside the SinglePostPage component:

pages/blog/[slug].tsx
import { allPosts } from "contentlayer/generated"
import { useMDXComponent } from "next-contentlayer/hooks"
import type { GetStaticProps, InferGetStaticPropsType } from "next"
import type { Post } from "contentlayer/generated"
 
export const getStaticPaths = () => {...}
 
export const getStaticProps = () => {...}
 
export default function SinglePostPage({
  post,
}: InferGetStaticPropsType<typeof getStaticProps>) {
  const MDXContent = useMDXComponent(post.body.code)
 
  return (
    <div>
      <h1>{post.title}</h1>
      <MDXContent />
    </div>
  )
}

There is a lot more to MDX components that we will explore in detail in the next post β€” including how to embed interactive components, re-map markdown constructs to custom React.js components and style your MDX content. Follow me on twitter for updates.

Closing Thoughts

This covers the majority of setup and configuration. As you've hopefully realized, Contentlayer greatly simplifies the process of implementing MDX.

We didn't have to choose and configure an mdx processor like mdx-bundler, we didn't use any file-system APIs or have to install third party glob or path packages, we didn't even have to think about ASTs or the complex unified, remark or rehype ecosystem.

On top of that, Contentlayer adds much more to the table like type-safety, convenience functions for dealing with your content and a single and structured place to define and customize your freeform data.

Further reading

β€’β€’β€’
Series
Build a Developer Blog with Next.js

  • A Fun and Productive Techstack to Build a Developer Blog
  • Turn Freeform MDX Content into Structured Data with Contentlayer
  • Add Interactivity and Flare to Markdown with MDX Components
  • Syntax Highlighting, the Easy and Performant Way, with Shiki and Rehype Pretty Code
  • Simple Post Metrics with useSWR and Prisma
  • Light and Dark Mode with Next Themes
  • Automated Open Graph Images with TBA
  • More Parts TBA