Better Transform Utilities for blitz/installer Recipes

[MrLeebo]

In my previous discussion, I approached the topic of generating an OpenAPI spec from your blitz app, and I assumed that there would have to be a recipe that you would run when you wanted to generate the spec. But when I attempted to build a proof of concept, I found myself really slowed down by the process of converting transformations into jscodeshift logic. To try to get a handle on it, I built a much simpler recipe blitz install secureheaders to get an understanding of how recipes are currently built and it wasn't to my liking.

So with that in mind, I thought I'd write up some of the things I'd like to see improved in this area.

Code Transformation Utility Function goals

The tools and utilities we have right now to write recipes are pretty rudimentary and I think they can be improved by committing to a set of utilities with the following properties:

• Terse
  It shouldn't take 20 lines of code just to toggle a boolean from true to false
• Safe
  Transform utilities shouldn't perform destructive actions if they have missing or loosely defined parameters (protection from rm -rf *-style transformations)
• Idempotent
  Transform utilities should be aware of what their transformation will do to the source code and should do nothing if the code already contains the target transformation. Calling addImport(program, importStatement) when the import is already in the file should not try to add it again
• Flexible
  Goes without saying, but if there isn't a utility that can perform the transformation you're trying to achieve, then you're back to square one

Specialized utilities may be needed to perform transformations on different languages, such as JSON, YML, and JSX.

There can also be utilities for common code transformations like adding statements to methods, setting some attributes on an object, or exporting a const. Ideally these utilities would avoid making you pass in jscodeshift expressions and will figure that out based on parameters as basic JavaScript types instead.

Next Step

I'd like to put together some code examples, but I am very out of time so that part will have to come later. Feel free to comment with utility ideas in the meantime!

[flybayer]

💯 💯 💯 💯 Absolutely, writing recipe DX can by WAY better. I'd love for you and others to work on it

[ntgussoni]

I've been through this while running blitz-guard-recipe. One thing that I wanted was to test the transformer, something 99% of the time want to do so I ended up including it in the recipe instead.

Making helpers could be an option, pretty much what eslint has for the most usual changes.

Another thing about recipe dx, it clones the whole repo, which means you cannot keep the recipe close to your code.
What if we allow to install as an npm package through npx or even a zip file on a github release?

[flybayer]

Yeah we can support npm

[MrLeebo]

OK I finally was able to jot down my list of transforms to start with. I've broken transforms down into four categories, JS, JSX, JSON, and schema.prisma.

JS transforms

This is the space that jscodeshift covers pretty well by itself and I'm not sure if a lot of recipes will be transforming JS files in ways that we can create basic utilities for. I think normally if a recipe has JS files, they'd be queries/mutations that are being added in addNewFilesStep() so you would just use template values to customize them then. But I dunno, maybe there's a use case I'm not thinking of.

JSON transforms

I don't think there really needs to be dedicated transforms for JSON files. They can already be updated without much trouble using plain JavaScript and JSON.parse:

// require node version 14 or greater
addTransformFilesStep({
  ...rest,
  singleFileSearch: paths.packageJson(),
  transformPlain(program: string) {
    const json = JSON.parse(program)
    json.engines.node = ">=14"
    return JSON.stringify(json, null, 2)
  }
})

JSX transforms

Extending JSX files will probably be the bread and butter of many recipes, but it also poses the most technical challenges. I think it would be awesome to be able to actually use JSX to define these types of transforms, but it would be really challenging to do that. For these examples I assumed that you can uniquely identify a component just given its name, but the real transform would probably need to accept some kind of selector object that can find a component based on a set of criteria (e.g. parent component, nth child, matching props, etc)

Some of the possible transforms in this category include:

• addDefaultImport
• addNamedImport
• wrapComponent
• setProps
• pushChild

I've included my estimate on what the function definitions for these transforms might look like, along with notes and some code examples.

/**
 * For the given source, add a default import to the program. If the source 
 * already has a default import, then the function will return the existing
 * identifier so that you can use it in your other transforms. If the import 
 * does not already exist, the defaultIdentifier will be used.
 * 
 * @example
 * const [program, identifier] = addDefaultImport(program, "react", "React")
 * identifier === "React"
 * 
 * import React from "react"
 */ 
function addDefaultImport(program: Collection<j.Program>, source: string, defaultIdentifier: string): [Collection<j.Program>, string]

/**
 * For the given source, add a named import to the program. Skip if the named
 * import already exists.
 * 
 * @example
 * addNamedImport(program, "react", "useEffect")
 * 
 * import { useEffect } from "react"
 */
function addNamedImport(program: Collection<j.Program>, source: string, identifier: string): Collection<j.Program>

/**
 * @todo I think we need some sort of smarter selector than just the string 
 * name of the component. May take some prototyping to figure out what this
 * might look like. Plus I'm not sure how to introduce props.
 * 
 * @example
 * <Widget />
 * 
 * wrapComponent(program, "Widget", "ErrorBoundary", { minimal: true })
 * 
 * <ErrorBoundary minimal>
 *   <Widget />
 * </ErrorBoundary>
 */
function wrapComponent(program: Collection<j.Program>, targetComponent: string, wrapComponent: string, props: any): Collection<j.Program>

/**
 * Finds the specified component and sets several props on it.
 * 
 * @example
 * <Widget />
 * 
 * setProps(program, "Widget", { debug: true, mode: "active", notYou: undefined, maxLength: 4 })
 * 
 * <Widget debug mode="active" maxLength={4} />
 */
function setProps(program: Collection<j.Program>, targetComponent: string, props: any): Collection<j.Program>

/**
 * Finds the specified component and adds a child component to the end of its 
 * children. If the component is self-closing, then open it up first.
 * 
 * @example
 * <Menu />
 * 
 * pushChild(program, "Menu", "Menu.Item", { key="special", disabled: true })
 * 
 * <Menu>
 *   <Menu.Item key="special" disabled />
 * </Menu>
 */
function pushChild(program: Collection<j.Program>, targetComponent: string, childComponent: string, props: any): Collection<j.Program>

OK, yeah that was a lot. Take a break, you've earned it. Your eyes are probably glazed over by this point and I'm betting most people have just scrolled past the rest of this comment by now. So if you're reading this and you didn't scroll ahead, give yourself a pat on the back for toughing it out! And leave a 🎉 reaction on the comment so I know you made it, you incredible human being!

schema.prisma transforms

A recipe might add a default model or other elements to your schema.prisma file so that you can try out the new functionality straight away after installing the recipe. schema.prisma doesn't have a whole lot of elements to it.

• setPrismaDataSource
• addPrismaGenerator
• addPrismaModel
• addPrismaField
• addPrismaIndex

Functions, notes, examples, you know how this goes:

/**
 * Update the datasource block of your schema.prisma file with the given 
 * provider and url.
 * @example
 * setPrismaDataSource(program, "postgres", `env("DATABASE_URL")`)
 * 
 * datasource db {
 *   provider = "postgres"
 *   url = env("DATABASE_URL")
 * }
 */
function setPrismaDataSource(program: string, provider: string, url: string): string

/**
 * Add a generator block to your schema.prisma file. Skips if there is 
 * already a generator by that name.
 * @example
 * addPrismaGenerator(program, "nexusPrisma", `provider = "nexus-prisma"`)
 * 
 * generator nexusPrisma {
 *   provider = "nexus-prisma"
 * }
 */
function addPrismaGenerator(program: string, generatorName: string, generatorBody: string): string

/**
 * Add a model definition to your schema.prisma file. Skips if there is already
 * a model by that name.
 * @example
 * addPrismaModel(program, "Project", `id Int @id @default(autoincrement())`)
 * 
 * model Project {
 *   id Int @id @default(autoincrement())}
 * }
 */
function addPrismaModel(program: string, modelName: string, modelBody: string): string

/**
 * Add a field to an existing prisma model. Skips if the model doesn't exist or
 * if a column with that name already exists.
 * @example
 * addPrismaField(program, "Project", "name", "String", "@unique")
 * 
 * model Project {
 *   id Int @id @default(autoincrement())
 *   name String @unique
 * }
 */
function addPrismaField(program: string, modelName: string, columnName: string, columnType: string, columnAttributes: string)

/**
 * Add an index to an existing prisma model. Skips if the model doesn't exist
 * or if an index with that name already exists.
 * @example
 * addPrismaIndex(program, "Project", "name_index", ["name"])
 * 
 * model Project {
 *   id Int @id @default(autoincrement())
 *   name String @unique
 *   @@index(name: "name_index", fields: [name])
 * }
 */
function addPrismaIndex(program: string, modelName: string, indexName: string, fields: string[]): string

None of these transforms would be very difficult to implement. They'll be like a nice 🍨 after the ordeal that was the JS/JSX transforms.

[flybayer]

This is awesome!

[MrLeebo]

Opened up #2169 to implement the schema.prisma transforms. I think it's better than what I originally had in mind because it actually parses the schema into an object you can modify, but on the other hand it's getting much more complicated than I originally expected 😅