Help me type my custom API with the schema

[kll]

Feature(s) impacted

Development with TypeScript.

Observed behavior

I’m having trouble with the following type:

interface MyType<
  S extends TSchema,
  N extends TCollectionName<S> = TCollectionName<S>,
  F extends TFieldName<S, N> = TFieldName<S, N>,
> {
  collectionName: N
  field: F
}

I would like to be able to dynamically use this type on any collection of my schema, like so:

const someValue: MyType<Schema> = {
  collectionName: 'User',
  field: 'Id',
}

(Schema is from typings.ts)
I am getting the following error on field:

TS2322: Type string is not assignable to type never
index. ts: The expected type comes from property field which is declared here on type MyType<Schema, keyof Schema, never>

That never at the end of my generics doesn’t look good…

If I attempt to specify N:

const someOtherValue: MyType<Schema, 'User'> = {
  collectionName: 'User',
  field: 'Id',
}

TypeScript doesn’t scream anymore, but I don’t see any relations fields! Only plain fields:

const someOtherValue: MyType<Schema, 'User'> = {
  collectionName: 'User',
  field: 'Organization:Id',
}

TS2322: Type ‘Organization:Id’ is not assignable to type TFieldName<Schema, ‘User’>
index. ts: The expected type comes from property field which is declared here on type
MyType<Schema, 'User', TFieldName<Schema, 'User'>>

So… how can I properly dynamically type these objects?

Expected behavior

My API can use the typings from typings.ts in a dynamic manner, allowing me to absolutely mess around with my schema with minimal boilerplate.

I’m trying to use this in the context of an in-house notification plugin. The plugin works magic, but I once had a notification crash because I messed up my projection ='D… so I’m trying to type it now.

In the end I’d like to be able to use the projections to type the callback parameters, so I don’t attempt to use a field I didn’t project.

Context

• Agent technology: Node
• Agent (forest package) name & version: @forestadmin/agent@1.55.1
• Database type: MSSQL

[dogan.ay]

Hello @kll

Always bringing fun topics to the support I see

I tried recreating the issue on my end but was not able to observe the same behaviour, did you set any particular TS linting rules and what IDE are you using ?

I defined the exact same interface as you did:

import { TCollectionName, TFieldName, TSchema } from "@forestadmin/agent"
import { Schema } from "./typings"

interface MyType<
  S extends TSchema,
  N extends TCollectionName<S> = TCollectionName<S>,
  F extends TFieldName<S, N> = TFieldName<S, N>,
> {
  collectionName: N
  field: F
}


const someOtherValue: MyType<Schema, 'users'> = {
    collectionName: 'users',
    field: 'machine:createdAt',
  }

With the following typings defined on my users collection

 'users': {
    plain: {
      'createdAt': string;
      'id': number;
      'id_machine': number;
      'lastName': string | null;
      'updatedAt': string;
      'username': string | null;
    };
    nested: {
      'machine': Schema['machines']['plain'] & Schema['machines']['nested'];
    };
    flat: {
      'machine:createdAt': string;
      'machine:id': number;
      'machine:name': string | null;
      'machine:updatedAt': string;
    };
  };

And as you can see, no issue on my VSCode IDE:

I was looking at the definition of TFieldName immediately suspecting the pipe

/** Field name (with relations) */
export type TFieldName<S extends TSchema, N extends TCollectionName<S>> = Extract<
  keyof S[N]['plain'] | keyof S[N]['flat'],
  string
>;

As some IDEs will behave weirdly with it.

What happens if you redefine the TFieldName type on your side and fiddle the definition, something like this maybe?

/** Field name (with relations) */
export type TFieldName<S extends TSchema, N extends TCollectionName<S>> = Extract<
  keyof (S[N]['plain'] & S[N]['flat']),
  string
>;

[kll]

@dogan.ay

Always bringing fun topics to the support I see

You know it =D I hope it’s somewhat of a breath of fresh air at least.
If there’s a more appropriate route for these requests, feel free to redirect me.
(inb4 no-reply@forestadmin.com)

Anyway. I use Webstorm, I tried on VSCode and got the same error. Maybe it’s something to do with the TS config… Do you have auto-completion on field?

No change with your definition of TFieldName.

Note that when adding segments, actions, and smart fields from a customizer, everything works perfectly (including autocompletion).

Here’s my tsconfig:

{
  "compilerOptions": {
    "lib": ["es2023"],
    "module": "CommonJS",
    "moduleResolution": "Node",
    "target": "ES2022",

    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,

    "forceConsistentCasingInFileNames": true,
    "pretty": true,
    "sourceMap": true,
    "outDir": "./dist",
    "baseUrl": ".",
    "types" : ["node"],
    "allowJs": false,
    // sequelize-typescript specific flags
    "experimentalDecorators": true,
    "emitDecoratorMetadata": true
  },
  "include": ["**/*", ".env"],
  "exclude": ["node_modules", "dist"]
}

[dogan.ay]

You know it =D I hope it’s somewhat of a breath of fresh air at least.
If there’s a more appropriate route for these requests, feel free to redirect me.

It’s perfect here

That’s weird, I do have autocompletion on mine.

I think I’m on to something, your interface declares three arguments but you only provide 2, the Schema and the collection name, what is the point of the third F ?

Would something like this work and answer your needs ?

interface MyType<
  S extends TSchema,
  N extends TCollectionName<S> = TCollectionName<S>,
> {
  collectionName: N
  field: TFieldName<S, N>
}

[kll]

Yeah using a third argument is just an artifact of my messy iterations.

Same issue:

TypeScript 5.7.2 btw

[dogan.ay]

Sorry I wasn’t quite catching the fact that you wanted to only pass the Schema to your interface. If I implement something similar (only the schema, I do have an error)

const someOtherValue: MyType<Schema> = {
    collectionName: 'users',
    field: 'machine:updatedAt',
  }

The autocompletion works when you define customization because you give the collectionName higher in the context of your customizeCollection

Generally functions allow to be more dynamic in the typings, I don’t know however if it is adapted to how your plugin expects its entries.

[kll]

I can work with a function.
Right now my plugin just uses a static array perks of being the sole user.

So you would use a function to build instances of MyType in that situation?

[dogan.ay]

I guess I was overly optimistic, I have not been able to achieve something without it being either too verbose or having to maintain a sort of registry. In which case I think passing the collection name might be better off.

One thing I realised is that you could just hard type the Schema in your interface as you would logically have only one, and do not need to be generic.
Meaning that you could get off with only having one typing parameter which would be the collection:

import { TCollectionName, TFieldName, TSchema } from "@forestadmin/agent";
import { Schema } from "./typings";

interface MyType<N extends TCollectionName<Schema>> {
  collectionName: TCollectionName<Schema>;
  field: TFieldName<Schema, N>;
}

const someValue: MyType<'users'> = {
  collectionName: 'users',
  field: 'machine:name',
}

[kll]

I think you were rightly optimistic because in the meantime, I got it working. Will post the solution once it’s less… more presentable.

[kll]

So. I had to throw away TFieldName and circumvent some Forest typing API, to end up with a result that actually has stronger typing!

It is, however, utterly cursed. I am never touching this again.

I don’t take any credit for all the witchery about to unfold:

• Dot Notation Type Accessor in TypeScript by Caleb Adepitan
• typescript - Transform union type to intersection type - Stack Overflow by jcalz

Here’s the setup:

// From https://www.calebpitan.com/blog/dot-notation-type-accessor-in-typescript
export type IsAny<T> = unknown extends T ? ([keyof T] extends [never] ? false : true) : false

type ExcludeArrayKeys<T> = T extends ArrayLike<any> ? Exclude<keyof T, keyof any[]> : keyof T

// Turned `.` into `:`
type PathImpl<T, Key extends keyof T> = Key extends string
  ? IsAny<T[Key]> extends true
    ? never
    : T[Key] extends Record<string, any>
      ?
      | `${Key}:${PathImpl<T[Key], ExcludeArrayKeys<T[Key]>> & string}`
      | `${Key}:${ExcludeArrayKeys<T[Key]> & string}`
      : never
  : never

type PathImpl2<T> = PathImpl<T, keyof T> | keyof T

type Path<T> = keyof T extends string
  ? PathImpl2<T> extends infer P
    ? P extends string | keyof T
      ? P
      : keyof T
    : keyof T
  : never

// Turned `.` into `:`
// Adapted to turn into a mapped type from https://www.calebpitan.com/blog/dot-notation-type-accessor-in-typescript
type Choose<
  T extends Record<string | number, any>,
  K extends Path<T>
> = K extends `${infer U}:${infer Rest}`
  ? Rest extends Path<T[U]>
    ? { [k in U]: Choose<T[U], Rest> }
    : never
  : { [k in K]: T[K] }

// From https://stackoverflow.com/a/50375286
type UnionToIntersection<U> =
  (U extends any ? (x: U)=>void : never) extends ((x: infer I)=>void) ? I : never

Breath in, breath out. I understand what’s going on thanks to the wonderful explanations from the authors above, but there’s no way I could write all of this.
There’s probably a dose of overkill there, as the schema structure is a lot simpler than what was initially planned by the author of the dot notation bonanza.

This allows us to create the following magic:

const typedItem = <N extends TCollectionName<Schema>>(collectionName: N) => <
  F extends Path<Schema[N]['plain'] & Schema[N]['nested']>,
  R extends UnionToIntersection<Choose<Schema[N]['plain'] & Schema[N]['nested'], F>>
>(o: {
  // Path<T> also returns intermediate paths, so filter them based on the schema
  fields: Extract<F, keyof Schema[N]['flat'] | keyof Schema[N]['plain']>[],
  callback(item: R): void
}) => ({
  collectionName,
  ...o
})

And finally, finally, get this beautifully typed call site!

typedItem('Session')({
  fields: ['Id', 'User:Banned'],
  callback(item) {
    console.log(item.Id, item.User.Banned)
  }
})

This is so beautiful. Auto-completion everywhere.

The whole goal of this endeavor was to avoid “dumb” issues. I’ve had silent bugs caused by undefined properties I forgot to add in the dependencies array. This is a recurring issue when defining smart actions and smart fields…

I hope this makes it through to official typings so we don’t just have a TRow<S, N>[] on callbacks in the future =)

[dogan.ay]

Amazing work @kll ! Thanks for the detailed result and the list of sources.
I’ll try to find time to read it all and as you said, it would be great to implement this for everyone.

I might be able to squeeze this during the next innovation session, otherwise I don’t really know how we could prioritise this.

If it gets close to being merged, I’ll co-author you and send you the PR =D