Switch to Typebox 1.x

[jmjf]

@sinclair/typebox@1.x introduces changes that break at least oneOf and possibly other outputs.

Evaluation

Migration guide

• ESM - already done that
• Kind, Hint, ReadonlyKind, OptionalKind replaced - affects oneOf
• Type.Date and Type.Uint8Array removed - not used (dates are based on Type.String())
• Type.Recursive replaced - not used
• Type.Regex removed - check b/c I'm not sure
• Type.Composite replaced - used in experiment
• Type.Transform renamed - used in experiment
• Type.Const removed in favor of Script - not used
• Type.Pick and Type.Omit changed - not used
• TypeCompiler changes - not used
• Referential type support - seems to be TypeCompiler and Transform related
• FormatRegistry moved - not used
• TypeRegistry removed in favor of Type.Base - affects oneOf
• TypeGuard removed - not used
• Value.Errors changes from iterator to array - not used
• Value.Cast renamed Repair - not used
• SetErrorFunction removed - not used
• CustomError handling - not used

To do

• oneOf needs changes for TypeRegistry and Kind changes in TypeBox 1.x
• Check how/if regex matters. Generally, I recommend avoiding OpenAPI regex formats in favor of known formats, which are more readable. API specs, like the museum spec, that use a pattern instead of format: time are painful to read in comparison.
• Update experiments that use Composite and Transform -- not critical, but might as well see if I can get them working.
  • Transform seems to be a simple rename to Type.Codec
  • Composite is replaced by Type.Interface and Type.Evaluate.

Notes

For use cases I'm most concerned with, I'm using AJV for validation, so code that uses output from to4ft doesn't care about changes to TypeBox's validator, validator error handling, validator formats. If you're using any of those TypeBox features, you need to update to TypeBox 1.x as part of moving to to4ft@2.x

to4ft doesn't use Type.Date because it works with the application/json media type where date and time data is a string and Fastify's JSON serializer. See #20, #21, #22, and #25 for previous work that ended with using Type.Unsafe<Date>() that supports passing a Date type to the serializer so Fastify will format it correctly while representing it as a string in input parameters. The application/json condition might change if #34 happens.

Items for later

• Revisit the concept of Type.Date and decide if it makes sense or not given our use case.

[jmjf]

oneOf

Going back to the original discussion, the first recommendation did not include the type registry. It just defined a OneOf function. The advantage of adding it to the type registry seems to be it's available to the type compiler, etc.

So, at a basic level, we're starting with something like:

const OneOf = <T extends TSchema[]>(oneOf: [...T], options: SchemaOptions = {}) => 
   Type.Unsafe<Static<TUnion<T>>>({ ... options, oneOf })

or, more like the migration guide style:

import { type TSchema, type SchemaOptions, type Static, Type } from '@sinclair/typebox';

function OneOf<T extends TSchema[]>(oneOf: [...T], options: SchemaOptions = {}) {
   return Type.Unsafe<Static<TUnion<T>>>({ ... options, oneOf });
}

Which should work without changes except maybe the import source. I don't think this validates that only one of the oneOf schemas is satisfied, as required by oneOf.

I think the TypeRegistry implementation might have bought that with the reduce check.

TypeRegistry.Set(
   'ExtendedOneOf',
   (schema: { oneOf: unknown[] }, value) =>
      schema.oneOf.reduce(
         (acc: number, schema: unknown) => acc + (Value.Check(schema as TSchema, value) ? 1 : 0),
         0,
      ) === 1,
);

const OneOf = <T extends TSchema[]>(oneOf: [...T], options: SchemaOptions = {}) =>
   Type.Unsafe<Static<TUnion<T>>>({ ...options, [Kind]: 'ExtendedOneOf', oneOf });

So, I think we end up with something kind of like below, but I'm sure this is broken. For one thing, Base requires a Clone in this case.

export class ExtendedOneOf extends Type.Base<'oneOf'> {
   public override Check(schema: {oneOf: unknown[] }, value:unknown): value is 'oneOf' {
      return schema.oneOf.reduce(acc: number, schema: unknown) => acc + (Value.Check(schema as TSchema, value) ? 1 : 0), 0, ) === 1
     // is there only one entry that matches?
}

export function OneOf<T extends TSchema[]>(oneOf: [...T], options: SchemaOptions = {}): ExtendedOneOf {
   return Type.Unsafe<Static<TUnion<T>>>({ ...options, ['~kind']: 'ExtendedOneOf', oneOf });
}

Are there use cases where having this as a proper TypeBox type would be helpful? Or could I just cheat with TUnion?

The Schema module has several oneOf bits, but I'm not sure they do what I want or how I'd use them.

Static can infer raw JSON schema. Is there a way to use this? For cases in code where I define TypeBox schemas, can I convert them to JSON schema structures and still feed them to AJV? In to4ft, this probably wouldn't support dereferencing well without a lot of work.

[jmjf]

TypeBox 1.0 notes

Type.Not is no more and there is no guidance on how to deal with that. It's valid JSON schema, but TypeBox can no longer construct it with Type.

Testing oneOf

This code produces the desired JSON Schema output for a oneOf.

import { type Static, Type, type TSchema, type TUnion, type TSchemaOptions } from 'typebox'

const OneOf = <T extends TSchema[]>(oneOf: [...T], options: TSchemaOptions = {}) => Type.Unsafe<Static<TUnion<T>>>({ ...options, ['~kind']: 'TO4FTOneOf', oneOf })

// ['~kind']: 'TO4FTOneOf' can be absent and still gets the same output

export const TestSchema = Type.Object( {
   title: Type.String({ default: 'hello', minLength: 3, maxLength: 100 }),
   testOneOf: Type.Optional(OneOf(
     [
        Type.Object({ a: Type.Number() }),
        Type.Object({ b: Type.String() }),
        Type.Array(Type.Object({ a: Type.String({ format: 'uuid' }) }))
     ]),
   )
});

export type Test = Static<typeof TestSchema>

console.log(TestSchema);
console.log(JSON.stringify(TestSchema, null, 3))

// t1 -> error for missing title
// const t1: Test = { testOneOf: { a: 1} }
//
// t2 -> error because fred is an unknown property
// const t2: Test = { title: 'something', fred: { a: 1 } }
//
// t3 -> error because c is not one of the three types defined
// const t3: Test = { title: 'something', testOneOf: { c: 'hello' } }
//
// t4 -> error because Date isn't assignable to string
// const t4: Test = { title: 'something', testOneOf: [{ a: new Date() }] }
//
// t5 -> no error because TypeScript doesn't check formats
// const t5: Test = { title: 'something', testOneOf: [{ a: Date.now().toString() }] }

Output

{
  type: 'object',
  required: [ 'title' ],
  properties: {
    title: { type: 'string', default: 'hello', minLength: 3, maxLength: 100 },
    testOneOf: { oneOf: [Array] }
  }
}
{
   "type": "object",
   "required": [
      "title"
   ],
   "properties": {
      "title": {
         "type": "string",
         "default": "hello",
         "minLength": 3,
         "maxLength": 100
      },
      "testOneOf": {
         "oneOf": [
            {
               "type": "object",
               "required": [
                  "a"
               ],
               "properties": {
                  "a": {
                     "type": "number"
                  }
               }
            },
            {
               "type": "object",
               "required": [
                  "b"
               ],
               "properties": {
                  "b": {
                     "type": "string"
                  }
               }
            },
            {
               "type": "array",
               "items": {
                  "type": "object",
                  "required": [
                     "a"
                  ],
                  "properties": {
                     "a": {
                        "type": "string",
                        "format": "uuid"
                     }
                  }
               }
            }
         ]
      }
   }
}

But output from the following is the same and errors reported are the same.

import { type Static } from 'typebox';

export const TestSchema = {
   type: 'object',
   properties: {
      title: { type: 'string', default: 'hello', minLength: 3, maxLength: 100 },
      testOneOf: {
         oneOf: [
            
              {
               type: 'object',
               properties: {
                 a: { type: 'number' },        
               },
               required: ['a']
            },
            { type: 'object', properties: { b: { type: 'string' } }, required: ['b'] },
            { type: 'array', items: { type: 'object', properties: { a: { type: 'string', format: 'uuid'} }, required: ['a']}}
       ]
    }
  },
  required: ['title'],
} as const;

export type Test = Static<typeof TestSchema>;

console.log(TestSchema);

console.log(JSON.stringify(TestSchema, null, 3));

Here, TestSchema is just JSON written as a plain old object. Using true JSON with " around everything inside the outermost { } gets the same result.

Something to think about...

The use case that motivated me to build to4ft is a project at work where we convert an OpenAPI spec for a Fastify-based API into:

• Fastify RouteOptions objects that let Fastify validate incoming request data with AJV and serialize results with Fastify's default serializer. These objects rely on JSON schemas built with TypeBox Type.* syntax.
• TypeScript types we use in controller code to assign types to request.query, request.body, etc., and to help us get response.body right. These types are just export type TheType = Static<TheSchema>.

If I can generate TheSchema as plain old JSON Schema, with some cleanup to remove parts I don't want to include because it's overhead, and if Static<TheSchema> still works, then maybe I should just write JSON Schema and add the type declaration and call it a day. to4ft no longer needs to convert JSON Schema to Type.* syntax, it just needs to clean up the JSON Schema.

Which is a breaking change, but since switching to TypeBox 1.0 is a major version bump anyway, that could be okay. Because the schema is JSON Schema, I won't need to find and work around all the issues with Type.* not having parts I need like Type.OneOf and Type.Not.

The main risks here are that TypeBox will break the JSON Schema interface (described as "evolving," though I think that's mainly if you're using TypeBox for validation) or remove it or I'll find some JSON Schema it doesn't handle that's common.

I think the next step is to dig back into tbCodeGenerators.ts and tbCodeParsers.ts and understand how they work together to see what needs to change. It may also be worth looking at openapi-transformer-toolkit's TSON generator code again and think about what that may tell me.