[Feature Request] MDX Support

[swyxio]

See #2064 for technical requirements.

Is your feature request related to a problem? Please describe.

I want to use MDX with NetlifyCMS. But I can't.

Describe the solution you'd like

NetlifyCMS should:

• detect my .mdx files
• not attempt to WYSIWYG my mdx files, but it can render in the right pane

Describe alternatives you've considered

nil. vuepress was mentioned in another issue

Additional context

some background chat done between me and @erquhart on the netlify internal slack as well as cms channel: https://netlify.slack.com/archives/C1C30FQFM/p1537919621000100

so i converted the gatsby netlifycms template to use mdx: https://gatsby-starter-netlify-cms-with-mdx.netlify.com/
source: https://github.com/sw-yx/gatsby-starter-netlify-cms-1
however netlifycms just totally fails to recognize mdx files instead of md files. i think the problem comes in config.yml where it is just hardcoded to parse markdown
in particular this line https://github.com/sw-yx/gatsby-starter-netlify-cms-1/blob/master/static/admin/config.yml#L11
what do you folks think we should do here?

@verythorough:

The collection field accepts an extension sub-field, but mdx is not a valid value for that right now:
https://www.netlifycms.org/docs/configuration-options/#collections
If it was added, we'd also need to add an mdx parser of some kind. Markdown is currently parsed/generated with Slate & Remark

@erquhart:

However we do it, MDX will require a fair amount of effort within the CMS itself.
Your work made me realize mdx has it's own extension - if that's always the case, we may be able to tackle a lot of this at the format level, before the file content ever reaches the editor.
To wit, the CMS reads in files and parses them before storing them in state, so maybe mdx files get some extra state pointing to the components they import, and those are then made available to the editor, but the markdown string that's passed to the editor is stripped of those import/export statements.

[swyxio]

one alternative i realized overnight is that we can write MDX files with an MD extension: https://gatsby-mdx.netlify.com/guides/mdx-file-extensions

so i did that to my own post: with these commits: https://github.com/sw-yx/gatsby-starter-netlify-cms-1/commit/07f6bd01d5f2133450c833deffba00a3107f9d99

and it worked! https://gatsby-starter-netlify-cms-with-mdx.netlify.com/blog/2017-01-04-this-is-an-mdx-post

Here's how it looks in netlify cms in markdown view

and the rich text view:

I can probably put in more work to fix the rendering on the right panel.

is this what we want? treat MDX as MD files? is that too hacky or do we want to have a special flow for MDX files?

[erquhart]

Hmm yeah I guess if the editor doesn't know about MDX it won't mess with any of it (already treats xml style markup as html and ignores). Really cool that MDX isn't 100% blocked for folks that really want it.

To announce support, however, we'll need preview support in place.

[tech4him1]

one alternative i realized overnight is that we can write MDX files with an MD extension

@sw-yx
Technically, you should be able to set on the collection:

extension: mdx
format: frontmatter

You can then leave the files with an mdx extension, and they will be treated like a regular MD file.

The collection field accepts an extension sub-field, but mdx is not a valid value for that right now:
https://www.netlifycms.org/docs/configuration-options/#collections

@verythorough That's not exactly correct -- the extension field will take any value (that's how people can use it for en.md, for example). The only difference if it is not one of those listed values is that the format must be set as well, since it cannot be inferred. I'm thinking we should update the linked documentation to make that more clear. Thoughts?

[verythorough]

@tech4him1 Right. The docs do actually say that in the two paragraphs that follow, but people (like me! and I wrote it!) might not read that far.

My Friday-afternoon brain isn't coming up with a clearer wording. Maybe we should file a separate issue for it. I'll drop it on my to-do list to look at later.

Another semi-related thing I think I'll make a feature request for is a "raw" format - something that does no parsing or previewing, but passes through the raw text, allows editing, and saves as typed. It's obviously not full support, but could be a nice option for allowing the authentication/browsing/saving flow of the CMS for formats that aren't supported (like rST, for example).

[swyxio]

yes. my kind of ideal (and i know i dont speak for everyone) is the dev.to workflow - where there is ONLY "raw" and then the preview pane. this way it's fast (because no wysiwyg) and extensible (mdx is a cinch). sorry i know this is classic scope creep :)

[swyxio]

inserting ok-mdx is an option https://github.com/jxnblk/ok-mdx

[talves]

@sw-yx is the only issue we have so far is to fix the preview (right pane)? or do you want full support from the editor also?

[swyxio]

yup, preview is fine.basically a dev.to workflow would be seamless to add this. but also there’s probably config stuff we need to work out.

[erquhart]

The big issue is MDX import and export statements.

ok-mdx looks awesome, great find @sw-yx! It may face similar issues as codemirror getting integrated into slate, but if nothing else it's a good starting point.

Another note for folks following this: MDX is now under the Unified umbrella along with Remark and Rehype, the libraries our markdown parsing is already driven by. We should be able to support MDX without wrapping in a purpose built editor, but hard to say for sure until someone digs in.

[talves]

I dug into it this weekend and rendering MDX during runtime for the preview is going to be a big ask.

[erquhart]

I expect it will look like this:

• pluck out import/exports before parsing the file, and prepare references to the imported/exported components
• add serialization logic to Remark for recognizing MDX imported React components
• I think that's all

From there, the markdown-turned-html and React components can all be static rendered via react-dom and live updated with any changes (which is what we already do). The question is where to register the React components that the MDX is importing. Probably really simple to solve, but I haven't spent any time considering it yet.

[karolis-sh]

I'm building an MDX based CMS at work, tried a few things and as @erquhart mentioned one of the main issues for me was the imports.
I made a small wrapper mdx-scoped-runtime around @mdx-js/runtime package that strips down the imports and layout export syntax using babel standalone. The gotcha is that you have to provide the possible importable components via scope prop for the pseudo-dynamic imports to work.

Also made a widget with said runtime. It's limited with the pre-defined scope prop, but the preview seems to work as expected.

I'd like to expand the mdx-scoped-runtime with ability to analyse the imports and provide some validation maybe, but the gatsby cms build has some issues (gatsbyjs/gatsby#10326) with certain babel plugins.

EDIT: I actually refactored the solution, turns out you can remove the imports via remark plugins quite easily, so no @babel/standalone required.

[damassi]

@buz-zard - the widget is working well 👌 It would be nice to find a way to make it work with gatsby-mdx (different package) global scope ChristopherBiscardi/gatsby-mdx#226, but for the time being the issue can be worked around by duplication.

[karolis-sh]

@damassi the mdx widget only uses the mdx-scoped-runtime. The CMS and gatsby pages are like 2 separate apps, so it really doesn't even matter what you use for gatsby+mdx handling.

Check the scoped runtime demo - it's just a minimal React hello-world app.

As for the global scope, check the readme - it's passed via scope prop.

I should probably extend the readme a bit 🤔

[damassi]

For those who are curious I have a full working starter example here , and here's the cms.js file.

[Undistraction]

@damassi Thanks. This looks excellent. Do all the gatsby-remark ... plugins work with mdx content? Other than the rendering can it be used interchangeably with plain markdown docs.

[damassi]

HI @Undistraction - You can pass in remark plugins at the gatsby-config.js level via gatsby-mdx. And yup, gatsby-mdx supports .md extensions as well.

Unrelated, it's important to note that when working with React components in .mdx and making edits in NetlifyCMS the globalScope key located just above does not apply to the scope required for NetlifyCMS: whatever is passed into one has to be passed into the other as well, e.g.,

// gatsby-config.js 

globalScope: `
  import *  as Components from 'rebass'
  export default { 
    ...Components 
  }
`

// cms.js
import * as Components from 'rebass'
...
scope: Components 

[Undistraction]

@damassi Thanks. This is helpful.

[talves]

@damassi tested the Button import out on the build and works now. Again, thanks.

I wonder how easy it would be to take this to a CRA solution.

[SachaG]

Trying to use gatsby-mdx, I'm getting an error that I think is related when doing gatsby develop:

 ERROR  Failed to compile with 1 errors                                                                                                               12:09:38

 error  in ./node_modules/gatsby-mdx/wrap-root-element.js

Module parse failed: Unexpected token (38:11)
You may need an appropriate loader to handle this file type.
|       ({ guard }) => (guard ? guard(props) : true)
|     );
>     return <Component {...props} />;
|   };
|

 @ ./node_modules/gatsby-mdx/gatsby-browser.js 1:0-39 2:31-35
 @ ./.cache/api-runner-browser-plugins.js
 @ ./.cache/api-runner-browser.js
 @ ./.cache/page-renderer.js
 @ ./.cache/json-store.js
 @ ./.cache/public-page-renderer-dev.js
 @ ./.cache/public-page-renderer.js
 @ ./.cache/gatsby-browser-entry.js
 @ ./src/components/Layout.js
 @ ./src/templates/about-page.js
 @ ./src/cms/preview-templates/AboutPagePreview.js
 @ ./src/cms/cms.js
 @ multi ./node_modules/gatsby-plugin-netlify-cms/cms.js ./src/cms/cms.js ./node_modules/gatsby-plugin-netlify-cms/cms-identity.js

Somehow wrap-root-element.js is not getting treated as jsx? Could this be related to NetlifyCMS or am I off the mark?

My gatsby info config:

  System:
    OS: macOS 10.14.3
    CPU: (8) x64 Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz
    Shell: 5.3 - /bin/zsh
  Binaries:
    Node: 8.12.0 - ~/.nvm/versions/node/v8.12.0/bin/node
    Yarn: 1.13.0 - /usr/local/bin/yarn
    npm: 6.7.0 - ~/.nvm/versions/node/v8.12.0/bin/npm
  Languages:
    Python: 3.6.4 - /usr/local/bin/python
  Browsers:
    Chrome: 73.0.3683.86
    Firefox: 65.0.1
    Safari: 12.0.3
  npmPackages:
    gatsby: ^2.1.37 => 2.1.37
    gatsby-image: ^2.0.34 => 2.0.34
    gatsby-mdx: ^0.4.4 => 0.4.4
    gatsby-plugin-netlify: ^2.0.13 => 2.0.13
    gatsby-plugin-netlify-cms: ^3.0.16 => 3.0.16
    gatsby-plugin-purgecss: ^3.1.0 => 3.1.0
    gatsby-plugin-react-helmet: ^3.0.10 => 3.0.10
    gatsby-plugin-sass: ^2.0.11 => 2.0.11
    gatsby-plugin-sharp: ^2.0.29 => 2.0.29
    gatsby-remark-copy-linked-files: ^2.0.11 => 2.0.11
    gatsby-remark-images: ^3.0.10 => 3.0.10
    gatsby-remark-relative-images: ^0.2.2 => 0.2.2
    gatsby-source-filesystem: ^2.0.27 => 2.0.27
    gatsby-source-graphql: ^2.0.15 => 2.0.15
    gatsby-transformer-remark: ^2.3.4 => 2.3.4
    gatsby-transformer-sharp: ^2.1.17 => 2.1.17
  npmGlobalPackages:
    gatsby-cli: 2.4.17

[erquhart]

@SachaG it isn't clear to me how that would be caused by Netlify CMS - have you raised an issue in the Gatsby repo?

[SachaG]

Not yet, I ended up putting MDX aside for now. But yeah, if nobody else is running into this then it's probably due to my Gatsby config.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mtliendo]

Really loving NetlifyCMS and would like to see this integrated.

If possible, can we get a list of blockers so that the community can try and work towards solving them?

Also, Is there an opportunity to collaborate with the folks over at contentful on this? It seems like they've had success in getting this implemented.

[erezrokah]

Hi @mtliendo, in the past months we've been trying to tackle the most upvoted features first:
https://github.com/netlify/netlify-cms/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc

Not sure whats is the definition for a blocker is in regards with new features.

Regrading collaboration, we would first need someone to take ownership of the issue and start working on it.

[mtliendo]

Hi @erezrokah 👋 Thanks for the link, and it looks like github projects are being used, so that gave me some awesome insight into where this is at. Regarding blockers, the issue seems like it's gone stale after trying to work out the preview screen and general error handling, but having specifics would help others pick up where @sw-yx left off.

Sorry, I'm new to the repo, are issues open to anyone in the community to own or are they championed by someone on the Netlify team?

[erezrokah]

Any issue is open for the community to contribute regardless of upvotes or prioritisation and will be backed by the Netlify team as much as possible :)
We mark some issues as good first issue, other ones might need some communication and discussion first.
Regardless of the issue it is always best to communicate first so we can assign an owner so people know the issue is taken.

If you'd like to work on this one let me know and I'll assign you.

[mtliendo]

Sorry didn't mean to ghost you 😅. I've been in the CMS all weekend and would like to take on some tasks, this one might be a bit more involved for my first one though. I'll keep an eye on it and others in the meantime and ping back when I spot something.

Thanks for all the info and direction!

[awilderink]

I got it working actually pretty nice together with React-Ace. Here my cms.jsx:

import React, { Component } from 'react'
import CMS from 'netlify-cms-app'
import IndexPagePreview from './preview-templates/IndexPagePreview'
import { components } from '../Theme'
import cloudinary from 'netlify-cms-media-library-cloudinary'
import withEmotion from './cssInjector'
import MDX from 'mdx-scoped-runtime'
import AceEditor from 'react-ace'
import 'ace-builds/src-noconflict/mode-jsx'
import 'ace-builds/src-noconflict/mode-html'
import 'ace-builds/src-noconflict/theme-monokai'

const isClient = typeof window !== 'undefined'
const isDevelopment = process.env.NODE_ENV === 'development'

if (isClient) {
  window.CMS_MANUAL_INIT = true
}

if (isDevelopment) {
  window.CMS_ENV = 'localhost_development'
}

class CodeEditor extends Component {
  render() {
    return (
      <AceEditor
        tabSize={2}
        mode="html" // or use jsx 
        theme="monokai"
        width="100%"
        value={this.props.value}
        onChange={this.props.onChange}
      />
    )
  }
}

CMS.registerPreviewTemplate('index', withEmotion(IndexPagePreview))

CMS.registerWidget(
  'mdx',
  CodeEditor,
  withEmotion(({ value }) => (
    <div className="netlify-cms-widget-mdx-preview">
      <MDX scope={components}>{value}</MDX>
    </div>
  ))
)

CMS.registerMediaLibrary(cloudinary)

CMS.init()

[narthur]

I would love support for MDX to be built in to the editor as well as the preview, since I'm considering using MDX on a project where non-technical folks need to feel comfortable editing pages.

[erezrokah]

@narthur, best way to push this it to upvote the issue since we use it for prioritisation: https://github.com/netlify/netlify-cms/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc

[nkuehn]

@awilderink thanks for sharing your approach, looks interesting! Can you explain how your "withEmotion" helper works (we're using styled components for all the MD(x) elements) and whether it would work with the standard editor instead of Ace, too? Any tips appreciated!

[awilderink]

@nkuehn Sorry for my late respons. The WithEmotion helper just inserts the prerendered CSS into the preview window. Here is the code:

import React from 'react'
import { CacheProvider } from '@emotion/core'
import createCache from '@emotion/cache'
import weakMemoize from '@emotion/weak-memoize'
import { ThemeProvider, Styled } from 'theme-ui'
import theme from '../gatsby-plugin-theme-ui'

const memoizedCreateCacheWithContainer = weakMemoize(container => {
  const newCache = createCache({ container })
  return newCache
})

export default Component => props => {
  const iframe = document.querySelector('#nc-root iframe')
  const iframeHeadElem = iframe && iframe.contentDocument.head

  if (!iframeHeadElem) {
    return null
  }

  return (
    <CacheProvider value={memoizedCreateCacheWithContainer(iframeHeadElem)}>
      <ThemeProvider theme={theme}>
        <Styled.root>
          <Component {...props} />
        </Styled.root>
      </ThemeProvider>
    </CacheProvider>
  )
}

I got it from somewhere else, but there are similar examples for styled-components too. I found a few examples here: #793

[karlhorky]

Also, Is there an opportunity to collaborate with the folks over at contentful on this? It seems like they've had success in getting this implemented.

@mtliendo seems like the code behind this example is actually not all that complicated:

https://github.com/ChristopherBiscardi/gatsby-mdx/blob/master/ui-extensions/contentful/index.js

So probably it wouldn't even need a collaboration - just someone to actually take the time to do the work here.

[karlhorky]

mdx-bundler by Kent C. Dodds may also be interesting here (uses xdm and esbuild for fast, powerful bundling): https://github.com/kentcdodds/mdx-bundler/

[dustinlacewell]

Starting a new project and curious about the status of this work.

[fabritw]

The below project may be of interest:

https://tina.io/blog/tina-supports-mdx/

[sivaraj-v]

Hi dev team, any update on this ? Good to hear some updates from you guys