Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tools.sendEvent() to step function tooling #75

Closed
wants to merge 41 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
34c2c71
Add async/await parallel steps, unify `createFunction`
jpwilliams Nov 13, 2022
d86673c
Use a normalised "frozen promise" for step tools for GC
jpwilliams Nov 23, 2022
521a6f6
Merge branch 'main' into async-await-parallel-steps
jpwilliams Nov 23, 2022
645a15c
Add `createFrozenPromise` helper
jpwilliams Dec 1, 2022
15c73f8
Merge branch 'main' into async-await-parallel-steps
jpwilliams Dec 13, 2022
bb6d71c
Merge branch 'main' into async-await-parallel-steps
jpwilliams Dec 14, 2022
7c9f475
Merge branch 'main' into async-await-parallel-steps
jpwilliams Dec 20, 2022
b87559b
Ordered promises progress dump
jpwilliams Dec 21, 2022
ff4f4a6
Adjust tests for new parallel strategies
jpwilliams Dec 21, 2022
9e88a7b
Remove unneeded code comments in `InngestStepTools`
jpwilliams Dec 21, 2022
b10c254
fix: Throw original error if serialization fails
jpwilliams Dec 21, 2022
9a914b4
Remove redundant comments/code after refactors
jpwilliams Dec 21, 2022
d848914
Merge branch 'main' into async-await-parallel-steps
jpwilliams Jan 6, 2023
8f3ea66
Fix `helloWorld` example to use client instead of non-existent helper
jpwilliams Jan 6, 2023
b850b6c
Refactor step fn internals to use hashes instead of built trees
jpwilliams Jan 6, 2023
574e72a
Add triggering steps via query parameters to comm handler and all han…
jpwilliams Jan 6, 2023
0de31e6
Fix function and step tooling tests to account for refactors
jpwilliams Jan 6, 2023
82dfabb
Push API diff for internal `InngestFunction`
jpwilliams Jan 6, 2023
46fd053
Add `StepPlanned` op code for new SDK behaviour
jpwilliams Jan 6, 2023
819b2c9
Refactor hashing to iterate position when finding duplicate IDs
jpwilliams Jan 9, 2023
d7d0a81
Use deterministic JSON stringify for hashing, replacing `sigmund`
jpwilliams Jan 9, 2023
130a1aa
Slight refactor of step function tests to be able to test hashing/pos
jpwilliams Jan 9, 2023
4c7eb7d
Fix `opts` being optional for `UnhashedOp` in step tooling
jpwilliams Jan 10, 2023
0579c2c
Remove reference to throwing `StepFlowInterrupt` errors
jpwilliams Jan 10, 2023
9a6c59a
Merge branch 'main' into async-await-parallel-steps
jpwilliams Jan 12, 2023
229134a
Fix logic and linting for `InngestStepTools` `sleepUntil` tests
jpwilliams Jan 12, 2023
802e2c0
Adjust received op stack to be similar to previous iterations
jpwilliams Jan 12, 2023
18b06b3
Merge branch 'main' into async-await-parallel-steps
jpwilliams Jan 14, 2023
d145d43
Add `tools.sendEvent` to step tooling to ensure events are duplicated
jpwilliams Jan 14, 2023
1970958
Fix `"inngest/digitalocean"` handler requiring `stepId` in `run` block
jpwilliams Jan 14, 2023
b44cd28
Merge branch 'async-await-parallel-steps' into feat/step-send-event
jpwilliams Jan 14, 2023
3bb9f8c
Internally rename `ReportStep`, always return array of ops, even runn…
jpwilliams Jan 16, 2023
6b2ddf2
Remove per-step error handling until executor support arrives
jpwilliams Jan 17, 2023
e2a926c
Handle differing executor data being sent between steps
jpwilliams Jan 17, 2023
efd8eaf
Merge branch 'main' into async-await-parallel-steps
jpwilliams Jan 17, 2023
908bb4f
Merge branch 'async-await-parallel-steps' into feat/step-send-event
jpwilliams Jan 17, 2023
be8acce
API diff from `main`
jpwilliams Jan 17, 2023
56d48aa
Merge branch 'async-await-parallel-steps' into feat/step-send-event
jpwilliams Jan 17, 2023
e298370
Alter a step's op code to `RunStep` when actually returning data
jpwilliams Jan 23, 2023
e7bffd3
Merge branch 'main' into async-await-parallel-steps
jpwilliams Jan 23, 2023
9f4f73b
Merge branch 'async-await-parallel-steps' into feat/step-send-event
jpwilliams Jan 23, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ Inngest allows you to:
<br />

<p align="center">
<a href="#getting-started">Getting started</a> ·
<a href="#features">Features</a> ·
<a href="#contributing">Contributing</a> ·
<a href="#getting-started">Getting started</a> ·
<a href="#features">Features</a> ·
<a href="#contributing">Contributing</a> ·
<a href="https://www.inngest.com/docs">Documentation</a>
</p>

Expand Down
80 changes: 15 additions & 65 deletions etc/inngest.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,29 +14,19 @@ export interface ClientOptions {
name: string;
}

// Warning: (ae-forgotten-export) The symbol "EventName" needs to be exported by the entry point index.d.ts
// Warning: (ae-forgotten-export) The symbol "InngestFunction" needs to be exported by the entry point index.d.ts
//
// @public
export const createFunction: <Event_1 extends EventPayload>(nameOrOpts: string | FunctionOptions, event: EventName<Event_1>, fn: SingleStepFn<Event_1, string, "step">) => InngestFunction<any>;

// @public
export const createScheduledFunction: (nameOrOpts: string | FunctionOptions, cron: string, fn: SingleStepFn<null, string, "step">) => InngestFunction<any>;

// @public
export const createStepFunction: <T extends EventPayload>(nameOrOpts: string | FunctionOptions, event: EventName<T>, fn: MultiStepFn<Record<T["name"], T>, T["name"], string, "step">) => InngestFunction<any>;

// @public
export interface EventPayload {
data: any;
name: string;
ts?: number;
user?: Record<string, any>;
user?: any;
v?: string;
}

// @public
export interface FunctionOptions {
// (undocumented)
fns?: Record<string, any>;
id?: string;
idempotency?: string;
name: string;
Expand All @@ -51,30 +41,14 @@ export interface FunctionOptions {
// @public
export class Inngest<Events extends Record<string, EventPayload>> {
constructor({ name, eventKey, inngestBaseUrl, fetch, }: ClientOptions);
createFunction<Event extends keyof Events, Name extends string, Fn extends SingleStepFn<Events[Event], Name, "step">>(
name: Name,
event: Event,
fn: Fn): InngestFunction<Events>;
createFunction<Event extends keyof Events, Opts extends FunctionOptions, Fn extends SingleStepFn<Events[Event], Opts extends FunctionOptions ? Opts["name"] : string, "step">>(
opts: Opts,
event: Event,
fn: Fn): InngestFunction<Events>;
createScheduledFunction<Name extends string>(
name: Name,
cron: string,
fn: SingleStepFn<null, Name, "step">): InngestFunction<Events>;
createScheduledFunction<Opts extends FunctionOptions>(
opts: Opts,
cron: string,
fn: SingleStepFn<null, Opts extends FunctionOptions ? Opts["name"] : string, "step">): InngestFunction<Events>;
createStepFunction<Event extends keyof Events, Name extends string, Fn extends MultiStepFn<Events, Event, Name, "step">>(
name: Name,
event: Event,
fn: Fn): InngestFunction<Events>;
createStepFunction<Event extends keyof Events, Opts extends FunctionOptions, Fn extends MultiStepFn<Events, Event, Opts extends FunctionOptions ? Opts["name"] : string, "step">>(
opts: Opts,
event: Event,
fn: Fn): InngestFunction<Events>;
// Warning: (ae-forgotten-export) The symbol "TriggerOptions" needs to be exported by the entry point index.d.ts
// Warning: (ae-forgotten-export) The symbol "Handler" needs to be exported by the entry point index.d.ts
// Warning: (ae-forgotten-export) The symbol "InngestFunction" needs to be exported by the entry point index.d.ts
//
// (undocumented)
createFunction<Trigger extends TriggerOptions<keyof Events & string>, NameOrOpts extends string | FunctionOptions>(nameOrOpts: NameOrOpts, trigger: Trigger, fn: Handler<Events, Trigger extends string ? Trigger : Trigger extends {
event: string;
} ? Trigger["event"] : string, NameOrOpts extends FunctionOptions ? NameOrOpts : never>): InngestFunction<Events>;
readonly inngestBaseUrl: URL;
readonly name: string;
// Warning: (ae-forgotten-export) The symbol "SingleOrArray" needs to be exported by the entry point index.d.ts
Expand All @@ -86,10 +60,10 @@ export class Inngest<Events extends Record<string, EventPayload>> {
eventKey: string): void;
}

// Warning: (ae-forgotten-export) The symbol "Handler" needs to be exported by the entry point index.d.ts
// Warning: (ae-forgotten-export) The symbol "Handler_2" needs to be exported by the entry point index.d.ts
//
// @public
export class InngestCommHandler<H extends Handler, TransformedRes> {
export class InngestCommHandler<H extends Handler_2, TransformedRes> {
constructor(
frameworkName: string,
appNameOrInngest: string | Inngest<any>,
Expand All @@ -116,7 +90,7 @@ export class InngestCommHandler<H extends Handler, TransformedRes> {
// Warning: (ae-forgotten-export) The symbol "StepRunResponse" needs to be exported by the entry point index.d.ts
//
// (undocumented)
protected runStep(functionId: string, stepId: string, data: any): Promise<StepRunResponse>;
protected runStep(functionId: string, stepId: string | null, data: any): Promise<StepRunResponse>;
// Warning: (ae-forgotten-export) The symbol "RegisterRequest" needs to be exported by the entry point index.d.ts
protected get sdkHeader(): [
prefix: string,
Expand All @@ -137,17 +111,6 @@ export class InngestCommHandler<H extends Handler, TransformedRes> {
protected validateSignature(): boolean;
}

// @public
export type MultiStepFn<Events extends Record<string, EventPayload>, Event extends keyof Events, FnId, StepId> = (arg: MultiStepFnArgs<Events, Event, FnId, StepId>) => void;

// @public
export interface MultiStepFnArgs<Events extends Record<string, EventPayload>, Event extends keyof Events, FnId, StepId> extends SingleStepFnArgs<Events[Event], FnId, StepId> {
// Warning: (ae-forgotten-export) The symbol "createStepTools" needs to be exported by the entry point index.d.ts
//
// (undocumented)
tools: ReturnType<typeof createStepTools<Events, Event>>[0];
}

// @public
export class NonRetriableError extends Error {
constructor(message: string, options?: {
Expand All @@ -173,20 +136,7 @@ functions: InngestFunction<any>[],
opts?: RegisterOptions) => any;

// @public
export type SingleStepFn<Event, FnId, StepId> = (arg: SingleStepFnArgs<Event, FnId, StepId>) => any;

// @public
export interface SingleStepFnArgs<Event, FnId, StepId> {
ctx: {
fn_id: FnId;
step_id: StepId;
};
event: Event;
steps: Record<string, never>;
}

// @public
export type TimeStr = `${`${number}w` | ""}${`${number}d` | ""}${`${number}h` | ""}${`${number}m` | ""}${`${number}s` | ""}${`${number}ms` | ""}`;
export type TimeStr = `${`${number}w` | ""}${`${number}d` | ""}${`${number}h` | ""}${`${number}m` | ""}${`${number}s` | ""}`;

// (No @packageDocumentation comment for this package)

Expand Down
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,9 @@
"cross-fetch": "^3.1.5",
"h3": "^1.0.2",
"hash.js": "^1.1.7",
"json-stringify-deterministic": "^1.0.8",
"ms": "^2.1.3",
"sigmund": "^1.0.1",
"serialize-error-cjs": "^0.1.3",
"type-fest": "^3.5.1",
"zod": "^3.19.1"
},
Expand All @@ -68,7 +69,6 @@
"@types/jest": "^27.4.1",
"@types/ms": "^0.7.31",
"@types/sha.js": "^2.4.0",
"@types/sigmund": "^1.0.0",
"@typescript-eslint/eslint-plugin": "^5.47.0",
"@typescript-eslint/parser": "^5.47.0",
"concurrently": "^7.4.0",
Expand Down
1 change: 1 addition & 0 deletions src/cloudflare.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ export const serve: ServeHandler = (nameOrInngest, fns, opts) => {
if (req.method === "POST") {
return {
fnId: url.searchParams.get(queryKeys.FnId) as string,
stepId: url.searchParams.get(queryKeys.StepId) as string,
data: (await req.json()) as Record<string, any>,
env,
isProduction,
Expand Down
Loading