Merge pull request #58 from bknd-io/release/0.6.2

Release 0.6.2

.gitignore

@@ -14,9 +14,11 @@ packages/media/.env
 **/*/.env
 **/*/.dev.vars
 **/*/.wrangler
+**/*/*.tgz
 **/*/vite.config.ts.timestamp*
 .history
 **/*/.db/*
+**/*/.configs/*
 **/*/*.db
 **/*/*.db-shm
 **/*/*.db-wal

LICENSE.md

@@ -6,7 +6,7 @@ FSL-1.1-MIT
 
 ## Notice
 
-Copyright 2024 Webintex GmbH
+Copyright 2025 Dennis Senn
 
 ## Terms and Conditions
 

README.md

@@ -1,9 +1,15 @@
-![bknd](https://raw.githubusercontent.com/bknd-io/bknd/refs/heads/main/app/src/ui/assets/poster.png)
+[![npm version](https://img.shields.io/npm/v/bknd.svg)](https://npmjs.org/package/bknd)
+[![npm downloads](https://img.shields.io/npm/dm/bknd)](https://www.npmjs.com/package/bknd)
 
-bknd simplifies app development by providing fully functional backend for data management, 
+![bknd](https://raw.githubusercontent.com/bknd-io/bknd/refs/heads/main/docs/_assets/poster.png)
-authentication, workflows and media. Since it's lightweight and built on Web Standards, it can 
+
-be deployed nearly anywhere, including running inside your framework of choice. No more 
+<p align="center" width="100%">
-deploying multiple separate services!
+<a href="https://stackblitz.com/github/bknd-io/bknd-examples?hideExplorer=1&embed=1&view=preview&startScript=example-admin-rich&initialPath=%2Fdata%2Fschema">
+<strong>⭐ Live Demo</strong>
+</a>
+</p>
+
+bknd simplifies app development by providing a fully functional backend for database management, authentication, media and workflows. Being lightweight and built on Web Standards, it can be deployed nearly anywhere, including running inside your framework of choice. No more deploying multiple separate services!
 
 **For documentation and examples, please visit https://docs.bknd.io.**
 
@@ -11,33 +17,116 @@ deploying multiple separate services!
 > Please keep in mind that **bknd** is still under active development
 > and therefore full backward compatibility is not guaranteed before reaching v1.0.0.
 
-## ✨ Features
+## Size
-**📊 Data**: Define, query, and control your data with ease. 
+![gzipped size of bknd](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/index.js?compression=gzip&label=bknd)
-  - Define entities with fields and relationships, synced directly to your database.  
+![gzipped size of bknd/client](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/ui/client/index.js?compression=gzip&label=bknd/client)
-  - Supported field types: `primary`, `text`, `number`, `date`, `boolean`, `enum`, `json`, `jsonschema`.  
+![gzipped size of bknd/elements](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/ui/elements/index.js?compression=gzip&label=bknd/elements)
-  - Relationship types: `one-to-one`, `many-to-one`, `many-to-many`, and `polymorphic`.  
+![gzipped size of bknd/ui](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/ui/index.js?compression=gzip&label=bknd/ui)
-  - Advanced querying with the **Repository**: filtering, sorting, pagination, and relational data handling.
-  - Seamlessly manage data with mutators and a robust event system.  
-  - Extend database functionality with batching, introspection, and support for multiple SQL dialects.
 
-**🔐 Auth**: Easily implement reliable authentication strategies.
+The size on npm is misleading, as the `bknd` package includes the backend, the ui components as well as the whole backend bundled into the cli including static assets. 
-  - Built-in `user` entity with customizable fields.  
-  - Supports multiple authentication strategies:  
-    - Email/password (with hashed storage).  
-    - OAuth/OIDC (Google, GitHub, and more).  
-  - Secure JWT generation and session management.
 
-**🖼️ Media**: Effortlessly manage and serve all your media files.
+## Motivation
-  - Upload files with ease.  
+Creating digital products always requires developing both the backend (the logic) and the frontend (the appearance). Building a backend from scratch demands deep knowledge in areas such as authentication and database management. Using a backend framework can speed up initial development, but it still requires ongoing effort to work within its constraints (e.g., *"how to do X with Y?"*), which can quickly slow you down. Choosing a backend system is a tough decision, as you might not be aware of its limitations until you encounter them.
-  - Adapter-based support for S3, S3-compatible storage (e.g., R2, Tigris), and Cloudinary.
 
-**🔄 Flows** (UI integration coming soon!): Design and run workflows with seamless automation.
+**The solution:** A backend system that only assumes and implements primitive details, integrates into multiple environments, and adheres to industry standards.
-  - Create and run workflows with trigger-based automation:  
+
-    - Manual triggers or events from data, auth, media, or server actions.  
+## Features
-    - HTTP triggers for external integrations.  
+* ⚡ Instant backend with full REST API:
-  - Define tasks in sequence, parallel, or loops, with conditional execution.  
+  * **Data**: Define, query, and control your data with ease.
-  - Use reusable sub-workflows to organize complex processes.  
+  * **Auth**: Easily implement reliable authentication strategies.
-  - Leverage OpenAPI specifications for API-based tasks.
+  * **Media**: Effortlessly manage and serve all your media files.
+  * **Flows**: Design and run workflows with seamless automation. (UI integration coming soon!)
+* 🌐 Built on Web Standards for maximum compatibility
+* 🏃‍♂️ Multiple run modes
+  * standalone using the CLI
+  * using a JavaScript runtime (Node, Bun, workerd)
+  * using a React framework (Astro, Remix, Next.js)
+* 📦 Official API and React SDK with type-safety
+* ⚛️ React elements for auto-configured authentication and media components
+
+## Structure
+The package is mainly split into 4 parts, each serving a specific purpose:
+
+| Import                      | Purpose                                              |
+|-----------------------------|------------------------------------------------------|
+| `bknd`<br/>`bknd/adapter/*` | Backend including APIs and adapters                  |
+| `bknd/ui`                   | Admin UI components for react frameworks             |
+| `bknd/client`               | TypeScript SDK and React hooks for the API endpoints |
+| `bknd/elements`             | React components for authentication and media        |
+
+
+### The backend (`bknd`)
+Serve the backend as an API for any JS runtime or framework. The latter is especially handy, as it allows you to deploy your frontend and backend bundled together. Furthermore it allows adding additional logic in a way you're already familar with. Just add another route and you're good to go.
+
+Here is an example of serving the API using node:
+```js index.js
+import { serve } from "bknd/adapter/node"
+serve();
+```
+
+### Integrated admin UI (`bknd/ui`)
+The admin UI allows to manage your data including full configuration of your backend using a graphical user interface. Using `vite`, your admin route looks like this:
+```tsx
+import { Admin } from "bknd/ui"
+import "bknd/dist/styles.css";
+
+export default function AdminPage() {
+   return <Admin />
+}
+```
+
+### Using the REST API or TypeScript SDK (`bknd/client`)
+If you're not using a JavaScript environment, you can still access any endpoint using the REST API:
+```bash
+curl -XGET <your-endpoint>/api/data/<entity>
+{
+  "data": [
+    { "id": 1, ... },
+    { "id": 2, ... }
+  ],
+  "meta": { /* ... */ }
+}
+```
+
+In a JavaScript environment, you can use the TypeScript SDK with type-safety. The above example would look like this:
+```ts
+import { Api } from "bknd/client";
+
+const api = new Api({ host: "<endpoint>" });
+const { data } = await api.data.readMany("<entity>");
+```
+
+If you're using React, there are 2 hooks exposed (`useApi`, `useEntity`), as well as an `swr` wrapper around each (`useApiQuery`, `useEntityQuery`). The `swr` wrapped hooks automatically handled query invalidation:
+
+```tsx
+import { useState } from "react";
+import { useEntityQuery } from "bknd/client";
+
+export default function App() {
+   const { data } = useEntityQuery("todos");   
+   return <ul>
+      {data?.map(todo => (
+         <li key={todo.id}>{todo.name}</li>
+      ))}
+   </ul>
+}
+```
+
+### React elements (`bknd/elements`)
+You don't have to figure out API details to include media uploads to your app. For an user avatar upload, this is all you need:
+```tsx
+import { Media } from "bknd/elements"
+import "bknd/dist/main.css"
+
+export function UserAvatar() {
+   return <Media.Dropzone
+     entity={{ name: "users", id: 1, field: "avatar" }}
+     maxItems={1}
+     overwrite
+   />
+}
+```
+The import path also exports components for login and registration forms which are automatically pointed to the `bknd` defaults.
 
 
 ## 🚀 Quick start

app/__test__/Module.spec.ts

@@ -1,38 +0,0 @@
-import { describe, expect, test } from "bun:test";
-import { type TSchema, Type, stripMark } from "../src/core/utils";
-import { Module } from "../src/modules/Module";
-
-function createModule<Schema extends TSchema>(schema: Schema) {
-   class TestModule extends Module<typeof schema> {
-      getSchema() {
-         return schema;
-      }
-      toJSON() {
-         return this.config;
-      }
-      useForceParse() {
-         return true;
-      }
-   }
-
-   return TestModule;
-}
-
-describe("Module", async () => {
-   test("basic", async () => {});
-
-   test("listener", async () => {
-      let result: any;
-
-      const module = createModule(Type.Object({ a: Type.String() }));
-      const m = new module({ a: "test" });
-
-      await m.schema().set({ a: "test2" });
-      m.setListener(async (c) => {
-         await new Promise((r) => setTimeout(r, 10));
-         result = stripMark(c);
-      });
-      await m.schema().set({ a: "test3" });
-      expect(result).toEqual({ a: "test3" });
-   });
-});

app/__test__/api/DataApi.spec.ts

@@ -0,0 +1,70 @@
+import { afterAll, beforeAll, describe, expect, it } from "bun:test";
+import { Guard } from "../../src/auth";
+import { parse } from "../../src/core/utils";
+import { DataApi } from "../../src/data/api/DataApi";
+import { DataController } from "../../src/data/api/DataController";
+import { dataConfigSchema } from "../../src/data/data-schema";
+import * as proto from "../../src/data/prototype";
+import { disableConsoleLog, enableConsoleLog, schemaToEm } from "../helper";
+
+beforeAll(disableConsoleLog);
+afterAll(enableConsoleLog);
+
+const dataConfig = parse(dataConfigSchema, {});
+describe("DataApi", () => {
+   it("should switch to post for long url reads", async () => {
+      const api = new DataApi();
+
+      const get = api.readMany("a".repeat(300), { select: ["id", "name"] });
+      expect(get.request.method).toBe("GET");
+      expect(new URL(get.request.url).pathname).toBe(`/api/data/${"a".repeat(300)}`);
+
+      const post = api.readMany("a".repeat(1000), { select: ["id", "name"] });
+      expect(post.request.method).toBe("POST");
+      expect(new URL(post.request.url).pathname).toBe(`/api/data/${"a".repeat(1000)}/query`);
+   });
+
+   it("returns result", async () => {
+      const schema = proto.em({
+         posts: proto.entity("posts", { title: proto.text() })
+      });
+      const em = schemaToEm(schema);
+      await em.schema().sync({ force: true });
+
+      const payload = [{ title: "foo" }, { title: "bar" }, { title: "baz" }];
+      await em.mutator("posts").insertMany(payload);
+
+      const ctx: any = { em, guard: new Guard() };
+      const controller = new DataController(ctx, dataConfig);
+      const app = controller.getController();
+
+      {
+         const res = (await app.request("/posts")) as Response;
+         const { data } = await res.json();
+         expect(data.length).toEqual(3);
+      }
+
+      // @ts-ignore tests
+      const api = new DataApi({ basepath: "/", queryLengthLimit: 50 });
+      // @ts-ignore protected
+      api.fetcher = app.request as typeof fetch;
+      {
+         const req = api.readMany("posts", { select: ["title"] });
+         expect(req.request.method).toBe("GET");
+         const res = await req;
+         expect(res.data).toEqual(payload);
+      }
+
+      {
+         const req = api.readMany("posts", {
+            select: ["title"],
+            limit: 100000,
+            offset: 0,
+            sort: "id"
+         });
+         expect(req.request.method).toBe("POST");
+         const res = await req;
+         expect(res.data).toEqual(payload);
+      }
+   });
+});

app/__test__/api/ModuleApi.spec.ts

@@ -0,0 +1,100 @@
+import { describe, expect, it } from "bun:test";
+import { Hono } from "hono";
+import { secureRandomString } from "../../src/core/utils";
+import { ModuleApi } from "../../src/modules";
+
+class Api extends ModuleApi {
+   _getUrl(path: string) {
+      return this.getUrl(path);
+   }
+}
+
+const host = "http://localhost";
+
+describe("ModuleApi", () => {
+   it("resolves options correctly", () => {
+      const api = new Api({ host });
+      expect(api.options).toEqual({ host });
+   });
+
+   it("returns correct url from path", () => {
+      const api = new Api({ host });
+      expect(api._getUrl("/test")).toEqual("http://localhost/test");
+      expect(api._getUrl("test")).toEqual("http://localhost/test");
+      expect(api._getUrl("test/")).toEqual("http://localhost/test");
+      expect(api._getUrl("//test?foo=1")).toEqual("http://localhost/test?foo=1");
+   });
+
+   it("fetches endpoint", async () => {
+      const app = new Hono().get("/endpoint", (c) => c.json({ foo: "bar" }));
+      const api = new Api({ host });
+
+      // @ts-expect-error it's protected
+      api.fetcher = app.request as typeof fetch;
+
+      const res = await api.get("/endpoint");
+      expect(res.res.ok).toEqual(true);
+      expect(res.res.status).toEqual(200);
+      expect(res.data).toEqual({ foo: "bar" });
+      expect(res.body).toEqual({ foo: "bar" });
+   });
+
+   it("has accessible request", async () => {
+      const app = new Hono().get("/endpoint", (c) => c.json({ foo: "bar" }));
+      const api = new Api({ host });
+
+      // @ts-expect-error it's protected
+      api.fetcher = app.request as typeof fetch;
+
+      const promise = api.get("/endpoint");
+      expect(promise.request).toBeDefined();
+      expect(promise.request.url).toEqual("http://localhost/endpoint");
+
+      expect((await promise).body).toEqual({ foo: "bar" });
+   });
+
+   it("adds token to headers when given in options", () => {
+      const token = secureRandomString(20);
+      const api = new Api({ host, token, token_transport: "header" });
+
+      expect(api.get("/").request.headers.get("Authorization")).toEqual(`Bearer ${token}`);
+   });
+
+   it("sets header to accept json", () => {
+      const api = new Api({ host });
+      expect(api.get("/").request.headers.get("Accept")).toEqual("application/json");
+   });
+
+   it("adds additional headers from options", () => {
+      const headers = new Headers({
+         "X-Test": "123"
+      });
+      const api = new Api({ host, headers });
+      expect(api.get("/").request.headers.get("X-Test")).toEqual("123");
+   });
+
+   it("uses basepath & removes trailing slash", () => {
+      const api = new Api({ host, basepath: "/api" });
+      expect(api.get("/").request.url).toEqual("http://localhost/api");
+   });
+
+   it("uses search params", () => {
+      const api = new Api({ host });
+      const search = new URLSearchParams({
+         foo: "bar"
+      });
+      expect(api.get("/", search).request.url).toEqual("http://localhost/?" + search.toString());
+   });
+
+   it("resolves method shortcut fns correctly", () => {
+      const api = new Api({ host });
+      expect(api.get("/").request.method).toEqual("GET");
+      expect(api.post("/").request.method).toEqual("POST");
+      expect(api.put("/").request.method).toEqual("PUT");
+      expect(api.patch("/").request.method).toEqual("PATCH");
+      expect(api.delete("/").request.method).toEqual("DELETE");
+   });
+
+   // @todo: test error response
+   // @todo: test method shortcut functions
+});

app/__test__/api/api.spec.ts

@@ -1,15 +0,0 @@
-import { describe, test } from "bun:test";
-import { DataApi } from "../../src/modules/data/api/DataApi";
-
-describe("Api", async () => {
-   test("...", async () => {
-      /*const dataApi = new DataApi({
-         host: "https://dev-config-soma.bknd.run"
-      });
-
-      const one = await dataApi.readOne("users", 1);
-      const many = await dataApi.readMany("users", { limit: 2 });
-      console.log("one", one);
-      console.log("many", many);*/
-   });
-});

app/__test__/app/repro.spec.ts

@@ -0,0 +1,73 @@
+import { describe, expect, test } from "bun:test";
+import { createApp, registries } from "../../src";
+import * as proto from "../../src/data/prototype";
+import { StorageLocalAdapter } from "../../src/media/storage/adapters/StorageLocalAdapter";
+
+describe("repros", async () => {
+   /**
+    * steps:
+    * 1. enable media
+    * 2. create 'test' entity
+    * 3. add media to 'test'
+    *
+    * There was an issue that AppData had old configs because of system entity "media"
+    */
+   test("registers media entity correctly to relate to it", async () => {
+      registries.media.register("local", StorageLocalAdapter);
+      const app = createApp();
+      await app.build();
+
+      {
+         // 1. enable media
+         const [, config] = await app.module.media.schema().patch("", {
+            enabled: true,
+            adapter: {
+               type: "local",
+               config: {
+                  path: "./"
+               }
+            }
+         });
+
+         expect(config.enabled).toBe(true);
+      }
+
+      {
+         // 2. create 'test' entity
+         await app.module.data.schema().patch(
+            "entities.test",
+            proto
+               .entity("test", {
+                  content: proto.text()
+               })
+               .toJSON()
+         );
+         expect(app.em.entities.map((e) => e.name)).toContain("test");
+      }
+
+      {
+         await app.module.data.schema().patch("entities.test.fields.files", {
+            type: "media",
+            config: {
+               required: false,
+               fillable: ["update"],
+               hidden: false,
+               mime_types: [],
+               virtual: true,
+               entity: "test"
+            }
+         });
+
+         expect(
+            app.module.data.schema().patch("relations.000", {
+               type: "poly",
+               source: "test",
+               target: "media",
+               config: { mappedBy: "files" }
+            })
+         ).resolves.toBeDefined();
+      }
+
+      expect(app.em.entities.map((e) => e.name)).toEqual(["media", "test"]);
+   });
+});

app/__test__/auth/middleware.spec.ts

@@ -0,0 +1 @@
+import { describe, expect, it } from "bun:test";

app/__test__/core/EventManager.spec.ts

@@ -1,8 +1,12 @@
-import { describe, expect, test } from "bun:test";
+import { afterAll, beforeAll, describe, expect, mock, test } from "bun:test";
-import { Event, EventManager, NoParamEvent } from "../../src/core/events";
+import { Event, EventManager, InvalidEventReturn, NoParamEvent } from "../../src/core/events";
+import { disableConsoleLog, enableConsoleLog } from "../helper";
+
+beforeAll(disableConsoleLog);
+afterAll(enableConsoleLog);
 
 class SpecialEvent extends Event<{ foo: string }> {
-   static slug = "special-event";
+   static override slug = "special-event";
 
    isBar() {
       return this.params.foo === "bar";
@@ -10,37 +14,139 @@ class SpecialEvent extends Event<{ foo: string }> {
 }
 
 class InformationalEvent extends NoParamEvent {
-   static slug = "informational-event";
+   static override slug = "informational-event";
+}
+
+class ReturnEvent extends Event<{ foo: string }, string> {
+   static override slug = "return-event";
+
+   override validate(value: string) {
+      if (typeof value !== "string") {
+         throw new InvalidEventReturn("string", typeof value);
+      }
+
+      return this.clone({
+         foo: [this.params.foo, value].join("-")
+      });
+   }
 }
 
 describe("EventManager", async () => {
-   test("test", async () => {
+   test("executes", async () => {
+      const call = mock(() => null);
+      const delayed = mock(() => null);
+
       const emgr = new EventManager();
       emgr.registerEvents([SpecialEvent, InformationalEvent]);
 
+      expect(emgr.eventExists("special-event")).toBe(true);
+      expect(emgr.eventExists("informational-event")).toBe(true);
+      expect(emgr.eventExists("unknown-event")).toBe(false);
+
       emgr.onEvent(
          SpecialEvent,
          async (event, name) => {
-            console.log("Event: ", name, event.params.foo, event.isBar());
+            expect(name).toBe("special-event");
-            console.log("wait...");
+            expect(event.isBar()).toBe(true);
-
+            call();
-            await new Promise((resolve) => setTimeout(resolve, 100));
+            await new Promise((resolve) => setTimeout(resolve, 50));
-            console.log("done waiting");
+            delayed();
          },
          "sync"
       );
 
+      // don't allow unknown
+      expect(() => emgr.on("unknown", () => void 0)).toThrow();
+
       emgr.onEvent(InformationalEvent, async (event, name) => {
-         console.log("Event: ", name, event.params);
+         call();
+         expect(name).toBe("informational-event");
       });
 
       await emgr.emit(new SpecialEvent({ foo: "bar" }));
-      console.log("done");
+      await emgr.emit(new InformationalEvent());
 
       // expect construct signatures to not cause ts errors
       new SpecialEvent({ foo: "bar" });
       new InformationalEvent();
 
-      expect(true).toBe(true);
+      expect(call).toHaveBeenCalledTimes(2);
+      expect(delayed).toHaveBeenCalled();
+   });
+
+   test("custom async executor", async () => {
+      const call = mock(() => null);
+      const asyncExecutor = (p: Promise<any>[]) => {
+         call();
+         return Promise.all(p);
+      };
+      const emgr = new EventManager(
+         { InformationalEvent },
+         {
+            asyncExecutor
+         }
+      );
+
+      emgr.onEvent(InformationalEvent, async () => {});
+      await emgr.emit(new InformationalEvent());
+      expect(call).toHaveBeenCalled();
+   });
+
+   test("piping", async () => {
+      const onInvalidReturn = mock(() => null);
+      const asyncEventCallback = mock(() => null);
+      const emgr = new EventManager(
+         { ReturnEvent, InformationalEvent },
+         {
+            onInvalidReturn
+         }
+      );
+
+      // @ts-expect-error InformationalEvent has no return value
+      emgr.onEvent(InformationalEvent, async () => {
+         asyncEventCallback();
+         return 1;
+      });
+
+      emgr.onEvent(ReturnEvent, async () => "1", "sync");
+      emgr.onEvent(ReturnEvent, async () => "0", "sync");
+
+      // @ts-expect-error must be string
+      emgr.onEvent(ReturnEvent, async () => 0, "sync");
+
+      // return is not required
+      emgr.onEvent(ReturnEvent, async () => {}, "sync");
+
+      // was "async", will not return
+      const e1 = await emgr.emit(new InformationalEvent());
+      expect(e1.returned).toBe(false);
+
+      const e2 = await emgr.emit(new ReturnEvent({ foo: "bar" }));
+      expect(e2.returned).toBe(true);
+      expect(e2.params.foo).toBe("bar-1-0");
+      expect(onInvalidReturn).toHaveBeenCalled();
+      expect(asyncEventCallback).toHaveBeenCalled();
+   });
+
+   test("once", async () => {
+      const call = mock(() => null);
+      const emgr = new EventManager({ InformationalEvent });
+
+      emgr.onEvent(
+         InformationalEvent,
+         async (event, slug) => {
+            expect(event).toBeInstanceOf(InformationalEvent);
+            expect(slug).toBe("informational-event");
+            call();
+         },
+         { mode: "sync", once: true }
+      );
+
+      expect(emgr.getListeners().length).toBe(1);
+      await emgr.emit(new InformationalEvent());
+      expect(emgr.getListeners().length).toBe(0);
+      await emgr.emit(new InformationalEvent());
+      expect(emgr.getListeners().length).toBe(0);
+      expect(call).toHaveBeenCalledTimes(1);
    });
 });

app/__test__/core/Registry.spec.ts

@@ -1,4 +1,4 @@
-import { describe, test } from "bun:test";
+import { describe, expect, test } from "bun:test";
 import type { TObject, TString } from "@sinclair/typebox";
 import { Registry } from "../../src/core/registry/Registry";
 import { type TSchema, Type } from "../../src/core/utils";
@@ -11,6 +11,9 @@ class What {
    method() {
       return null;
    }
+   getType() {
+      return Type.Object({ type: Type.String() });
+   }
 }
 class What2 extends What {}
 class NotAllowed {}
@@ -32,25 +35,53 @@ describe("Registry", () => {
       } satisfies Record<string, Test1>);
 
       const item = registry.get("first");
+      expect(item).toBeDefined();
+      expect(item?.cls).toBe(What);
 
+      const second = Type.Object({ type: Type.String(), what: Type.String() });
       registry.add("second", {
          cls: What2,
-         schema: Type.Object({ type: Type.String(), what: Type.String() }),
+         schema: second,
          enabled: true
       });
+      // @ts-ignore
+      expect(registry.get("second").schema).toEqual(second);
+
+      const third = Type.Object({ type: Type.String({ default: "1" }), what22: Type.String() });
       registry.add("third", {
          // @ts-expect-error
          cls: NotAllowed,
-         schema: Type.Object({ type: Type.String({ default: "1" }), what22: Type.String() }),
+         schema: third,
          enabled: true
       });
+      // @ts-ignore
+      expect(registry.get("third").schema).toEqual(third);
+
+      const fourth = Type.Object({ type: Type.Number(), what22: Type.String() });
       registry.add("fourth", {
          cls: What,
          // @ts-expect-error
-         schema: Type.Object({ type: Type.Number(), what22: Type.String() }),
+         schema: fourth,
          enabled: true
       });
+      // @ts-ignore
+      expect(registry.get("fourth").schema).toEqual(fourth);
 
-      console.log("list", registry.all());
+      expect(Object.keys(registry.all()).length).toBe(4);
+   });
+
+   test("uses registration fn", async () => {
+      const registry = new Registry<Test1>((a: ClassRef<What>) => {
+         return {
+            cls: a,
+            schema: a.prototype.getType(),
+            enabled: true
+         };
+      });
+
+      registry.register("what2", What2);
+      expect(registry.get("what2")).toBeDefined();
+      expect(registry.get("what2").cls).toBe(What2);
+      expect(registry.get("what2").schema).toEqual(What2.prototype.getType());
    });
 });

app/__test__/core/crypto.spec.ts

@@ -1,14 +1,16 @@
-import { describe, test } from "bun:test";
+import { describe, expect, test } from "bun:test";
 import { checksum, hash } from "../../src/core/utils";
 
 describe("crypto", async () => {
    test("sha256", async () => {
-      console.log(await hash.sha256("test"));
+      expect(await hash.sha256("test")).toBe(
+         "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
+      );
    });
    test("sha1", async () => {
-      console.log(await hash.sha1("test"));
+      expect(await hash.sha1("test")).toBe("a94a8fe5ccb19ba61c4c0873d391e987982fbbd3");
    });
    test("checksum", async () => {
-      console.log(checksum("hello world"));
+      expect(await checksum("hello world")).toBe("2aae6c35c94fcfb415dbe95f408b9ce91ee846ed");
    });
 });

app/__test__/data/data-query-impl.spec.ts

@@ -1,16 +1,21 @@
 import { describe, expect, test } from "bun:test";
-import type { QueryObject } from "ufo";
+import { Value, _jsonp } from "../../src/core/utils";
-import { WhereBuilder, type WhereQuery } from "../../src/data/entities/query/WhereBuilder";
+import { type RepoQuery, WhereBuilder, type WhereQuery, querySchema } from "../../src/data";
+import type { RepoQueryIn } from "../../src/data/server/data-query-impl";
 import { getDummyConnection } from "./helper";
 
-const t = "t";
+const decode = (input: RepoQueryIn, expected: RepoQuery) => {
+   const result = Value.Decode(querySchema, input);
+   expect(result).toEqual(expected);
+};
+
 describe("data-query-impl", () => {
    function qb() {
       const c = getDummyConnection();
       const kysely = c.dummyConnection.kysely;
-      return kysely.selectFrom(t).selectAll();
+      return kysely.selectFrom("t").selectAll();
    }
-   function compile(q: QueryObject) {
+   function compile(q: WhereQuery) {
       const { sql, parameters } = WhereBuilder.addClause(qb(), q).compile();
       return { sql, parameters };
    }
@@ -89,4 +94,47 @@ describe("data-query-impl", () => {
          expect(keys).toEqual(expectedKeys);
       }
    });
+
+   test("with", () => {
+      decode({ with: ["posts"] }, { with: { posts: {} } });
+      decode({ with: { posts: {} } }, { with: { posts: {} } });
+      decode({ with: { posts: { limit: 1 } } }, { with: { posts: { limit: 1 } } });
+      decode(
+         {
+            with: {
+               posts: {
+                  with: {
+                     images: {
+                        select: ["id"]
+                     }
+                  }
+               }
+            }
+         },
+         {
+            with: {
+               posts: {
+                  with: {
+                     images: {
+                        select: ["id"]
+                     }
+                  }
+               }
+            }
+         }
+      );
+   });
+});
+
+describe("data-query-impl: Typebox", () => {
+   test("sort", async () => {
+      const _dflt = { sort: { by: "id", dir: "asc" } };
+
+      decode({ sort: "" }, _dflt);
+      decode({ sort: "name" }, { sort: { by: "name", dir: "asc" } });
+      decode({ sort: "-name" }, { sort: { by: "name", dir: "desc" } });
+      decode({ sort: "-posts.name" }, { sort: { by: "posts.name", dir: "desc" } });
+      decode({ sort: "-1name" }, _dflt);
+      decode({ sort: { by: "name", dir: "desc" } }, { sort: { by: "name", dir: "desc" } });
+   });
 });

app/__test__/data/data.test.ts

@@ -18,7 +18,7 @@ describe("some tests", async () => {
 
    const users = new Entity("users", [
       new TextField("username", { required: true, default_value: "nobody" }),
-      new TextField("email", { max_length: 3 })
+      new TextField("email", { maxLength: 3 })
    ]);
 
    const posts = new Entity("posts", [

app/__test__/data/mutation.simple.test.ts

@@ -16,7 +16,7 @@ describe("Mutator simple", async () => {
       new TextField("label", { required: true, minLength: 1 }),
       new NumberField("count", { default_value: 0 })
    ]);
-   const em = new EntityManager([items], connection);
+   const em = new EntityManager<any>([items], connection);
 
    await em.connection.kysely.schema
       .createTable("items")
@@ -175,4 +175,18 @@ describe("Mutator simple", async () => {
          { id: 8, label: "keep", count: 0 }
       ]);
    });
+
+   test("insertMany", async () => {
+      const oldCount = (await em.repo(items).count()).count;
+      const inserts = [{ label: "insert 1" }, { label: "insert 2" }];
+      const { data } = await em.mutator(items).insertMany(inserts);
+
+      expect(data.length).toBe(2);
+      expect(data.map((d) => ({ label: d.label }))).toEqual(inserts);
+      const newCount = (await em.repo(items).count()).count;
+      expect(newCount).toBe(oldCount + inserts.length);
+
+      const { data: data2 } = await em.repo(items).findMany({ offset: oldCount });
+      expect(data2).toEqual(data);
+   });
 });

app/__test__/data/prototype.test.ts

@@ -3,6 +3,8 @@ import {
    BooleanField,
    DateField,
    Entity,
+   EntityIndex,
+   EntityManager,
    EnumField,
    JsonField,
    ManyToManyRelation,
@@ -12,6 +14,7 @@ import {
    PolymorphicRelation,
    TextField
 } from "../../src/data";
+import { DummyConnection } from "../../src/data/connection/DummyConnection";
 import {
    FieldPrototype,
    type FieldSchema,
@@ -20,6 +23,7 @@ import {
    boolean,
    date,
    datetime,
+   em,
    entity,
    enumm,
    json,
@@ -46,12 +50,17 @@ describe("prototype", () => {
    });
 
    test("...2", async () => {
-      const user = entity("users", {
+      const users = entity("users", {
-         name: text().required(),
+         name: text(),
          bio: text(),
          age: number(),
-         some: number().required()
+         some: number()
       });
+      type db = {
+         users: Schema<typeof users>;
+      };
+
+      const obj: Schema<typeof users> = {} as any;
 
       //console.log("user", user.toJSON());
    });
@@ -266,4 +275,38 @@ describe("prototype", () => {
 
       const obj: Schema<typeof test> = {} as any;
    });
+
+   test("schema", async () => {
+      const _em = em(
+         {
+            posts: entity("posts", { name: text(), slug: text().required() }),
+            comments: entity("comments", { some: text() }),
+            users: entity("users", { email: text() })
+         },
+         ({ relation, index }, { posts, comments, users }) => {
+            relation(posts).manyToOne(comments).manyToOne(users);
+            index(posts).on(["name"]).on(["slug"], true);
+         }
+      );
+
+      type LocalDb = (typeof _em)["DB"];
+
+      const es = [
+         new Entity("posts", [new TextField("name"), new TextField("slug", { required: true })]),
+         new Entity("comments", [new TextField("some")]),
+         new Entity("users", [new TextField("email")])
+      ];
+      const _em2 = new EntityManager(
+         es,
+         new DummyConnection(),
+         [new ManyToOneRelation(es[0], es[1]), new ManyToOneRelation(es[0], es[2])],
+         [
+            new EntityIndex(es[0], [es[0].field("name")!]),
+            new EntityIndex(es[0], [es[0].field("slug")!], true)
+         ]
+      );
+
+      // @ts-ignore
+      expect(_em2.toJSON()).toEqual(_em.toJSON());
+   });
 });

app/__test__/data/relations.test.ts

@@ -106,7 +106,6 @@ describe("Relations", async () => {
       expect(postAuthorRel?.other(posts).entity).toBe(users);
 
       const kysely = em.connection.kysely;
-      const jsonFrom = (e) => e;
       /**
        * Relation Helper
        */
@@ -119,14 +118,11 @@ describe("Relations", async () => {
        - select: users.*
        - cardinality: 1
        */
-      const selectPostsFromUsers = postAuthorRel.buildWith(
+      const selectPostsFromUsers = kysely
-         users,
+         .selectFrom(users.name)
-         kysely.selectFrom(users.name),
+         .select((eb) => postAuthorRel.buildWith(users, "posts")(eb).as("posts"));
-         jsonFrom,
-         "posts"
-      );
       expect(selectPostsFromUsers.compile().sql).toBe(
-         'select (select "posts"."id" as "id", "posts"."title" as "title", "posts"."author_id" as "author_id" from "posts" as "posts" where "posts"."author_id" = "users"."id" limit ?) as "posts" from "users"'
+         'select (select from "posts" as "posts" where "posts"."author_id" = "users"."id") as "posts" from "users"'
       );
       expect(postAuthorRel!.getField()).toBeInstanceOf(RelationField);
       const userObj = { id: 1, username: "test" };
@@ -141,15 +137,12 @@ describe("Relations", async () => {
        - select: posts.*
        - cardinality:
        */
-      const selectUsersFromPosts = postAuthorRel.buildWith(
+      const selectUsersFromPosts = kysely
-         posts,
+         .selectFrom(posts.name)
-         kysely.selectFrom(posts.name),
+         .select((eb) => postAuthorRel.buildWith(posts, "author")(eb).as("author"));
-         jsonFrom,
-         "author"
-      );
 
       expect(selectUsersFromPosts.compile().sql).toBe(
-         'select (select "author"."id" as "id", "author"."username" as "username" from "users" as "author" where "author"."id" = "posts"."author_id" limit ?) as "author" from "posts"'
+         'select (select from "users" as "author" where "author"."id" = "posts"."author_id" limit ?) as "author" from "posts"'
       );
       expect(postAuthorRel.getField()).toBeInstanceOf(RelationField);
       const postObj = { id: 1, title: "test" };
@@ -315,20 +308,16 @@ describe("Relations", async () => {
        - select: users.*
        - cardinality: 1
        */
-      const selectCategoriesFromPosts = postCategoriesRel.buildWith(
+      const selectCategoriesFromPosts = kysely
-         posts,
+         .selectFrom(posts.name)
-         kysely.selectFrom(posts.name),
+         .select((eb) => postCategoriesRel.buildWith(posts)(eb).as("categories"));
-         jsonFrom
-      );
       expect(selectCategoriesFromPosts.compile().sql).toBe(
          'select (select "categories"."id" as "id", "categories"."label" as "label" from "categories" inner join "posts_categories" on "categories"."id" = "posts_categories"."categories_id" where "posts"."id" = "posts_categories"."posts_id" limit ?) as "categories" from "posts"'
       );
 
-      const selectPostsFromCategories = postCategoriesRel.buildWith(
+      const selectPostsFromCategories = kysely
-         categories,
+         .selectFrom(categories.name)
-         kysely.selectFrom(categories.name),
+         .select((eb) => postCategoriesRel.buildWith(categories)(eb).as("posts"));
-         jsonFrom
-      );
       expect(selectPostsFromCategories.compile().sql).toBe(
          'select (select "posts"."id" as "id", "posts"."title" as "title" from "posts" inner join "posts_categories" on "posts"."id" = "posts_categories"."posts_id" where "categories"."id" = "posts_categories"."categories_id" limit ?) as "posts" from "categories"'
       );

app/__test__/data/specs/Mutator.spec.ts

@@ -1,4 +1,5 @@
 import { afterAll, describe, expect, test } from "bun:test";
+import type { EventManager } from "../../../src/core/events";
 import {
    Entity,
    EntityManager,
@@ -10,6 +11,7 @@ import {
    RelationMutator,
    TextField
 } from "../../../src/data";
+import * as proto from "../../../src/data/prototype";
 import { getDummyConnection } from "../helper";
 
 const { dummyConnection, afterAllCleanup } = getDummyConnection();
@@ -22,7 +24,7 @@ describe("[data] Mutator (base)", async () => {
       new TextField("hidden", { hidden: true }),
       new TextField("not_fillable", { fillable: false })
    ]);
-   const em = new EntityManager([entity], dummyConnection);
+   const em = new EntityManager<any>([entity], dummyConnection);
    await em.schema().sync({ force: true });
 
    const payload = { label: "item 1", count: 1 };
@@ -61,7 +63,7 @@ describe("[data] Mutator (ManyToOne)", async () => {
    const posts = new Entity("posts", [new TextField("title")]);
    const users = new Entity("users", [new TextField("username")]);
    const relations = [new ManyToOneRelation(posts, users)];
-   const em = new EntityManager([posts, users], dummyConnection, relations);
+   const em = new EntityManager<any>([posts, users], dummyConnection, relations);
    await em.schema().sync({ force: true });
 
    test("RelationMutator", async () => {
@@ -83,14 +85,12 @@ describe("[data] Mutator (ManyToOne)", async () => {
 
       // persisting reference should ...
       expect(
-         postRelMutator.persistReference(relations[0], "users", {
+         postRelMutator.persistReference(relations[0]!, "users", {
             $set: { id: userData.data.id }
          })
       ).resolves.toEqual(["users_id", userData.data.id]);
       // @todo: add what methods are allowed to relation, like $create should not be allowed for post<>users
 
-      process.exit(0);
-
       const userRelMutator = new RelationMutator(users, em);
       expect(userRelMutator.getRelationalKeys()).toEqual(["posts"]);
    });
@@ -99,7 +99,7 @@ describe("[data] Mutator (ManyToOne)", async () => {
       expect(
          em.mutator(posts).insertOne({
             title: "post1",
-            users_id: 1 // user does not exist yet
+            users_id: 100 // user does not exist yet
          })
       ).rejects.toThrow();
    });
@@ -192,7 +192,7 @@ describe("[data] Mutator (OneToOne)", async () => {
    const users = new Entity("users", [new TextField("username")]);
    const settings = new Entity("settings", [new TextField("theme")]);
    const relations = [new OneToOneRelation(users, settings)];
-   const em = new EntityManager([users, settings], dummyConnection, relations);
+   const em = new EntityManager<any>([users, settings], dummyConnection, relations);
    await em.schema().sync({ force: true });
 
    test("insertOne: missing ref", async () => {
@@ -276,7 +276,7 @@ describe("[data] Mutator (ManyToMany)", async () => {
 
 describe("[data] Mutator (Events)", async () => {
    const entity = new Entity("test", [new TextField("label")]);
-   const em = new EntityManager([entity], dummyConnection);
+   const em = new EntityManager<any>([entity], dummyConnection);
    await em.schema().sync({ force: true });
    const events = new Map<string, any>();
 
@@ -299,4 +299,71 @@ describe("[data] Mutator (Events)", async () => {
       expect(events.has(MutatorEvents.MutatorDeleteBefore.slug)).toBeTrue();
       expect(events.has(MutatorEvents.MutatorDeleteAfter.slug)).toBeTrue();
    });
+
+   test("insertOne event return is respected", async () => {
+      const posts = proto.entity("posts", {
+         title: proto.text(),
+         views: proto.number()
+      });
+
+      const conn = getDummyConnection();
+      const em = new EntityManager([posts], conn.dummyConnection);
+      await em.schema().sync({ force: true });
+
+      const emgr = em.emgr as EventManager<any>;
+
+      emgr.onEvent(
+         // @ts-ignore
+         EntityManager.Events.MutatorInsertBefore,
+         async (event) => {
+            return {
+               ...event.params.data,
+               views: 2
+            };
+         },
+         "sync"
+      );
+
+      const mutator = em.mutator("posts");
+      const result = await mutator.insertOne({ title: "test", views: 1 });
+      expect(result.data).toEqual({
+         id: 1,
+         title: "test",
+         views: 2
+      });
+   });
+
+   test("updateOne event return is respected", async () => {
+      const posts = proto.entity("posts", {
+         title: proto.text(),
+         views: proto.number()
+      });
+
+      const conn = getDummyConnection();
+      const em = new EntityManager([posts], conn.dummyConnection);
+      await em.schema().sync({ force: true });
+
+      const emgr = em.emgr as EventManager<any>;
+
+      emgr.onEvent(
+         // @ts-ignore
+         EntityManager.Events.MutatorUpdateBefore,
+         async (event) => {
+            return {
+               ...event.params.data,
+               views: event.params.data.views + 1
+            };
+         },
+         "sync"
+      );
+
+      const mutator = em.mutator("posts");
+      const created = await mutator.insertOne({ title: "test", views: 1 });
+      const result = await mutator.updateOne(created.data.id, { views: 2 });
+      expect(result.data).toEqual({
+         id: 1,
+         title: "test",
+         views: 3
+      });
+   });
 });

app/__test__/data/specs/Repository.spec.ts

@@ -1,7 +1,6 @@
 import { afterAll, describe, expect, test } from "bun:test";
-// @ts-ignore
-import { Perf } from "@bknd/core/utils";
 import type { Kysely, Transaction } from "kysely";
+import { Perf } from "../../../src/core/utils";
 import {
    Entity,
    EntityManager,
@@ -24,7 +23,7 @@ async function sleep(ms: number) {
 }
 
 describe("[Repository]", async () => {
-   test("bulk", async () => {
+   test.skip("bulk", async () => {
       //const connection = dummyConnection;
       //const connection = getLocalLibsqlConnection();
       const credentials = null as any; // @todo: determine what to do here

app/__test__/data/specs/WithBuilder.spec.ts

@@ -1,4 +1,5 @@
 import { afterAll, describe, expect, test } from "bun:test";
+import { _jsonp } from "../../../src/core/utils";
 import {
    Entity,
    EntityManager,
@@ -8,19 +9,56 @@ import {
    TextField,
    WithBuilder
 } from "../../../src/data";
+import * as proto from "../../../src/data/prototype";
+import { compileQb, prettyPrintQb, schemaToEm } from "../../helper";
 import { getDummyConnection } from "../helper";
 
-const { dummyConnection, afterAllCleanup } = getDummyConnection();
+const { dummyConnection } = getDummyConnection();
-afterAll(afterAllCleanup);
 
 describe("[data] WithBuilder", async () => {
+   test("validate withs", async () => {
+      const schema = proto.em(
+         {
+            posts: proto.entity("posts", {}),
+            users: proto.entity("users", {}),
+            media: proto.entity("media", {})
+         },
+         ({ relation }, { posts, users, media }) => {
+            relation(posts).manyToOne(users);
+            relation(users).polyToOne(media, { mappedBy: "avatar" });
+         }
+      );
+      const em = schemaToEm(schema);
+
+      expect(WithBuilder.validateWiths(em, "posts", undefined)).toBe(0);
+      expect(WithBuilder.validateWiths(em, "posts", {})).toBe(0);
+      expect(WithBuilder.validateWiths(em, "posts", { users: {} })).toBe(1);
+      expect(
+         WithBuilder.validateWiths(em, "posts", {
+            users: {
+               with: { avatar: {} }
+            }
+         })
+      ).toBe(2);
+      expect(() => WithBuilder.validateWiths(em, "posts", { author: {} })).toThrow();
+      expect(() =>
+         WithBuilder.validateWiths(em, "posts", {
+            users: {
+               with: { glibberish: {} }
+            }
+         })
+      ).toThrow();
+   });
+
    test("missing relation", async () => {
       const users = new Entity("users", [new TextField("username")]);
       const em = new EntityManager([users], dummyConnection);
 
       expect(() =>
-         WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, ["posts"])
+         WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, {
-      ).toThrow('Relation "posts" not found');
+            posts: {}
+         })
+      ).toThrow('Relation "users<>posts" not found');
    });
 
    test("addClause: ManyToOne", async () => {
@@ -29,36 +67,39 @@ describe("[data] WithBuilder", async () => {
       const relations = [new ManyToOneRelation(posts, users, { mappedBy: "author" })];
       const em = new EntityManager([users, posts], dummyConnection, relations);
 
-      const qb = WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, [
+      const qb = WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, {
-         "posts"
+         posts: {}
-      ]);
+      });
 
       const res = qb.compile();
 
       expect(res.sql).toBe(
-         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'content\', "agg"."content", \'author_id\', "agg"."author_id")), \'[]\') from (select "posts"."id" as "id", "posts"."content" as "content", "posts"."author_id" as "author_id" from "posts" where "users"."id" = "posts"."author_id" limit ?) as agg) as "posts" from "users"'
+         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'content\', "agg"."content", \'author_id\', "agg"."author_id")), \'[]\') from (select "posts"."id" as "id", "posts"."content" as "content", "posts"."author_id" as "author_id" from "posts" as "posts" where "posts"."author_id" = "users"."id" order by "posts"."id" asc limit ? offset ?) as agg) as "posts" from "users"'
       );
-      expect(res.parameters).toEqual([5]);
+      expect(res.parameters).toEqual([10, 0]);
 
       const qb2 = WithBuilder.addClause(
          em,
          em.connection.kysely.selectFrom("posts"),
          posts, // @todo: try with "users", it gives output!
-         ["author"]
+         {
+            author: {}
+         }
       );
 
       const res2 = qb2.compile();
 
       expect(res2.sql).toBe(
-         'select (select json_object(\'id\', "obj"."id", \'username\', "obj"."username") from (select "users"."id" as "id", "users"."username" as "username" from "users" where "posts"."author_id" = "users"."id" limit ?) as obj) as "author" from "posts"'
+         'select (select json_object(\'id\', "obj"."id", \'username\', "obj"."username") from (select "users"."id" as "id", "users"."username" as "username" from "users" as "author" where "author"."id" = "posts"."author_id" order by "users"."id" asc limit ? offset ?) as obj) as "author" from "posts"'
       );
-      expect(res2.parameters).toEqual([1]);
+      expect(res2.parameters).toEqual([1, 0]);
    });
 
    test("test with empty join", async () => {
+      const em = new EntityManager([], dummyConnection);
       const qb = { qb: 1 } as any;
 
-      expect(WithBuilder.addClause(null as any, qb, null as any, [])).toBe(qb);
+      expect(WithBuilder.addClause(em, qb, null as any, {})).toBe(qb);
    });
 
    test("test manytomany", async () => {
@@ -89,7 +130,7 @@ describe("[data] WithBuilder", async () => {
 
       //console.log((await em.repository().findMany("posts_categories")).result);
 
-      const res = await em.repository(posts).findMany({ with: ["categories"] });
+      const res = await em.repository(posts).findMany({ with: { categories: {} } });
 
       expect(res.data).toEqual([
          {
@@ -107,7 +148,7 @@ describe("[data] WithBuilder", async () => {
          }
       ]);
 
-      const res2 = await em.repository(categories).findMany({ with: ["posts"] });
+      const res2 = await em.repository(categories).findMany({ with: { posts: {} } });
 
       //console.log(res2.sql, res2.data);
 
@@ -121,8 +162,8 @@ describe("[data] WithBuilder", async () => {
             id: 2,
             label: "beauty",
             posts: [
-               { id: 2, title: "beauty post" },
+               { id: 1, title: "fashion post" },
-               { id: 1, title: "fashion post" }
+               { id: 2, title: "beauty post" }
             ]
          },
          {
@@ -150,25 +191,25 @@ describe("[data] WithBuilder", async () => {
          em,
          em.connection.kysely.selectFrom("categories"),
          categories,
-         ["single"]
+         { single: {} }
       );
       const res = qb.compile();
       expect(res.sql).toBe(
-         'select (select json_object(\'id\', "obj"."id", \'path\', "obj"."path") from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" limit ?) as obj) as "single" from "categories"'
+         'select (select json_object(\'id\', "obj"."id", \'path\', "obj"."path") from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" order by "media"."id" asc limit ? offset ?) as obj) as "single" from "categories"'
       );
-      expect(res.parameters).toEqual(["categories.single", 1]);
+      expect(res.parameters).toEqual(["categories.single", 1, 0]);
 
       const qb2 = WithBuilder.addClause(
          em,
          em.connection.kysely.selectFrom("categories"),
          categories,
-         ["multiple"]
+         { multiple: {} }
       );
       const res2 = qb2.compile();
       expect(res2.sql).toBe(
-         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'path\', "agg"."path")), \'[]\') from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" limit ?) as agg) as "multiple" from "categories"'
+         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'path\', "agg"."path")), \'[]\') from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" order by "media"."id" asc limit ? offset ?) as agg) as "multiple" from "categories"'
       );
-      expect(res2.parameters).toEqual(["categories.multiple", 5]);
+      expect(res2.parameters).toEqual(["categories.multiple", 10, 0]);
    });
 
    /*test("test manytoone", async () => {
@@ -192,4 +233,205 @@ describe("[data] WithBuilder", async () => {
       const res = await em.repository().findMany("posts", { join: ["author"] });
       console.log(res.sql, res.parameters, res.result);
    });*/
+
+   describe("recursive", () => {
+      test("compiles with singles", async () => {
+         const schema = proto.em(
+            {
+               posts: proto.entity("posts", {}),
+               users: proto.entity("users", {
+                  username: proto.text()
+               }),
+               media: proto.entity("media", {
+                  path: proto.text()
+               })
+            },
+            ({ relation }, { posts, users, media }) => {
+               relation(posts).manyToOne(users);
+               relation(users).polyToOne(media, { mappedBy: "avatar" });
+            }
+         );
+         const em = schemaToEm(schema);
+
+         const qb = WithBuilder.addClause(
+            em,
+            em.connection.kysely.selectFrom("posts"),
+            schema.entities.posts,
+            {
+               users: {
+                  limit: 5, // ignored
+                  select: ["id", "username"],
+                  sort: { by: "username", dir: "asc" },
+                  with: {
+                     avatar: {
+                        select: ["id", "path"],
+                        limit: 2 // ignored
+                     }
+                  }
+               }
+            }
+         );
+
+         //prettyPrintQb(qb);
+         expect(qb.compile().sql).toBe(
+            'select (select json_object(\'id\', "obj"."id", \'username\', "obj"."username", \'avatar\', "obj"."avatar") from (select "users"."id" as "id", "users"."username" as "username", (select json_object(\'id\', "obj"."id", \'path\', "obj"."path") from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "users"."id" = "media"."entity_id" order by "media"."id" asc limit ? offset ?) as obj) as "avatar" from "users" as "users" where "users"."id" = "posts"."users_id" order by "users"."username" asc limit ? offset ?) as obj) as "users" from "posts"'
+         );
+         expect(qb.compile().parameters).toEqual(["users.avatar", 1, 0, 1, 0]);
+      });
+
+      test("compiles with many", async () => {
+         const schema = proto.em(
+            {
+               posts: proto.entity("posts", {}),
+               comments: proto.entity("comments", {}),
+               users: proto.entity("users", {
+                  username: proto.text()
+               }),
+               media: proto.entity("media", {
+                  path: proto.text()
+               })
+            },
+            ({ relation }, { posts, comments, users, media }) => {
+               relation(posts).manyToOne(users).polyToOne(media, { mappedBy: "images" });
+               relation(users).polyToOne(media, { mappedBy: "avatar" });
+               relation(comments).manyToOne(posts).manyToOne(users);
+            }
+         );
+         const em = schemaToEm(schema);
+
+         const qb = WithBuilder.addClause(
+            em,
+            em.connection.kysely.selectFrom("posts"),
+            schema.entities.posts,
+            {
+               comments: {
+                  limit: 12,
+                  with: {
+                     users: {
+                        select: ["username"]
+                     }
+                  }
+               }
+            }
+         );
+
+         expect(qb.compile().sql).toBe(
+            'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'posts_id\', "agg"."posts_id", \'users_id\', "agg"."users_id", \'users\', "agg"."users")), \'[]\') from (select "comments"."id" as "id", "comments"."posts_id" as "posts_id", "comments"."users_id" as "users_id", (select json_object(\'username\', "obj"."username") from (select "users"."username" as "username" from "users" as "users" where "users"."id" = "comments"."users_id" order by "users"."id" asc limit ? offset ?) as obj) as "users" from "comments" as "comments" where "comments"."posts_id" = "posts"."id" order by "comments"."id" asc limit ? offset ?) as agg) as "comments" from "posts"'
+         );
+         expect(qb.compile().parameters).toEqual([1, 0, 12, 0]);
+      });
+
+      test("returns correct result", async () => {
+         const schema = proto.em(
+            {
+               posts: proto.entity("posts", {
+                  title: proto.text()
+               }),
+               comments: proto.entity("comments", {
+                  content: proto.text()
+               }),
+               users: proto.entity("users", {
+                  username: proto.text()
+               }),
+               media: proto.entity("media", {
+                  path: proto.text()
+               })
+            },
+            ({ relation }, { posts, comments, users, media }) => {
+               relation(posts).manyToOne(users).polyToOne(media, { mappedBy: "images" });
+               relation(users).polyToOne(media, { mappedBy: "avatar" });
+               relation(comments).manyToOne(posts).manyToOne(users);
+            }
+         );
+         const em = schemaToEm(schema);
+         await em.schema().sync({ force: true });
+
+         // add data
+         await em.mutator("users").insertMany([{ username: "user1" }, { username: "user2" }]);
+         await em.mutator("posts").insertMany([
+            { title: "post1", users_id: 1 },
+            { title: "post2", users_id: 1 },
+            { title: "post3", users_id: 2 }
+         ]);
+         await em.mutator("comments").insertMany([
+            { content: "comment1", posts_id: 1, users_id: 1 },
+            { content: "comment1-1", posts_id: 1, users_id: 1 },
+            { content: "comment2", posts_id: 1, users_id: 2 },
+            { content: "comment3", posts_id: 2, users_id: 1 },
+            { content: "comment4", posts_id: 2, users_id: 2 },
+            { content: "comment5", posts_id: 3, users_id: 1 },
+            { content: "comment6", posts_id: 3, users_id: 2 }
+         ]);
+
+         const result = await em.repo("posts").findMany({
+            select: ["title"],
+            with: {
+               comments: {
+                  limit: 2,
+                  select: ["content"],
+                  with: {
+                     users: {
+                        select: ["username"]
+                     }
+                  }
+               }
+            }
+         });
+
+         expect(result.data).toEqual([
+            {
+               title: "post1",
+               comments: [
+                  {
+                     content: "comment1",
+                     users: {
+                        username: "user1"
+                     }
+                  },
+                  {
+                     content: "comment1-1",
+                     users: {
+                        username: "user1"
+                     }
+                  }
+               ]
+            },
+            {
+               title: "post2",
+               comments: [
+                  {
+                     content: "comment3",
+                     users: {
+                        username: "user1"
+                     }
+                  },
+                  {
+                     content: "comment4",
+                     users: {
+                        username: "user2"
+                     }
+                  }
+               ]
+            },
+            {
+               title: "post3",
+               comments: [
+                  {
+                     content: "comment5",
+                     users: {
+                        username: "user1"
+                     }
+                  },
+                  {
+                     content: "comment6",
+                     users: {
+                        username: "user2"
+                     }
+                  }
+               ]
+            }
+         ]);
+         //console.log(_jsonp(result.data));
+      });
+   });
 });

app/__test__/data/specs/fields/EnumField.spec.ts

@@ -13,10 +13,6 @@ describe("[data] EnumField", async () => {
       { options: options(["a", "b", "c"]) }
    );
 
-   test("yields if no options", async () => {
-      expect(() => new EnumField("test", { options: options([]) })).toThrow();
-   });
-
    test("yields if default value is not a valid option", async () => {
       expect(
          () => new EnumField("test", { options: options(["a", "b"]), default_value: "c" })

app/__test__/data/specs/fields/Field.spec.ts

@@ -15,11 +15,9 @@ describe("[data] Field", async () => {
 
    runBaseFieldTests(FieldSpec, { defaultValue: "test", schemaType: "text" });
 
-   test.only("default config", async () => {
+   test("default config", async () => {
-      const field = new FieldSpec("test");
       const config = Default(baseFieldConfigSchema, {});
       expect(stripMark(new FieldSpec("test").config)).toEqual(config);
-      console.log("config", new TextField("test", { required: true }).toJSON());
    });
 
    test("transformPersist (specific)", async () => {

app/__test__/data/specs/fields/JsonField.spec.ts

@@ -32,7 +32,7 @@ describe("[data] JsonField", async () => {
    });
 
    test("getValue", async () => {
-      expect(field.getValue({ test: 1 }, "form")).toBe('{"test":1}');
+      expect(field.getValue({ test: 1 }, "form")).toBe('{\n  "test": 1\n}');
       expect(field.getValue("string", "form")).toBe('"string"');
       expect(field.getValue(1, "form")).toBe("1");
 

app/__test__/data/specs/relations/EntityRelation.spec.ts

@@ -70,9 +70,9 @@ describe("[data] EntityRelation", async () => {
 
    it("required", async () => {
       const relation1 = new TestEntityRelation();
-      expect(relation1.config.required).toBe(false);
+      expect(relation1.required).toBe(false);
 
       const relation2 = new TestEntityRelation({ required: true });
-      expect(relation2.config.required).toBe(true);
+      expect(relation2.required).toBe(true);
    });
 });

app/__test__/flows/workflow-basic.test.ts

@@ -1,7 +1,7 @@
 // eslint-disable-next-line import/no-unresolved
 import { describe, expect, test } from "bun:test";
 import { isEqual } from "lodash-es";
-import { type Static, Type, _jsonp } from "../../src/core/utils";
+import { type Static, Type, _jsonp, withDisabledConsole } from "../../src/core/utils";
 import { Condition, ExecutionEvent, FetchTask, Flow, LogTask, Task } from "../../src/flows";
 
 /*beforeAll(disableConsoleLog);
@@ -232,7 +232,9 @@ describe("Flow tests", async () => {
       ).toEqual(["second", "fourth"]);
 
       const execution = back.createExecution();
-      expect(execution.start()).rejects.toThrow();
+      withDisabledConsole(async () => {
+         expect(execution.start()).rejects.toThrow();
+      });
    });
 
    test("Flow with back step: enough retries", async () => {

app/__test__/helper.ts

@@ -1,7 +1,9 @@
 import { unlink } from "node:fs/promises";
-import type { SqliteDatabase } from "kysely";
+import type { SelectQueryBuilder, SqliteDatabase } from "kysely";
 import Database from "libsql";
-import { SqliteLocalConnection } from "../src/data";
+import { format as sqlFormat } from "sql-formatter";
+import { type Connection, EntityManager, SqliteLocalConnection } from "../src/data";
+import type { em as protoEm } from "../src/data/prototype";
 
 export function getDummyDatabase(memory: boolean = true): {
    dummyDb: SqliteDatabase;
@@ -40,7 +42,7 @@ const _oldConsoles = {
    error: console.error
 };
 
-export function disableConsoleLog(severities: ConsoleSeverity[] = ["log"]) {
+export function disableConsoleLog(severities: ConsoleSeverity[] = ["log", "warn"]) {
    severities.forEach((severity) => {
       console[severity] = () => null;
    });
@@ -51,3 +53,18 @@ export function enableConsoleLog() {
       console[severity as ConsoleSeverity] = fn;
    });
 }
+
+export function compileQb(qb: SelectQueryBuilder<any, any, any>) {
+   const { sql, parameters } = qb.compile();
+   return { sql, parameters };
+}
+
+export function prettyPrintQb(qb: SelectQueryBuilder<any, any, any>) {
+   const { sql, parameters } = qb.compile();
+   console.log("$", sqlFormat(sql), "\n[params]", parameters);
+}
+
+export function schemaToEm(s: ReturnType<typeof protoEm>, conn?: Connection): EntityManager<any> {
+   const connection = conn ? conn : getDummyConnection().dummyConnection;
+   return new EntityManager(Object.values(s.entities), connection, s.relations, s.indices);
+}

app/__test__/integration/auth.integration.test.ts

@@ -0,0 +1,213 @@
+import { afterAll, beforeAll, describe, expect, it } from "bun:test";
+import { App, createApp } from "../../src";
+import type { AuthResponse } from "../../src/auth";
+import { randomString, secureRandomString, withDisabledConsole } from "../../src/core/utils";
+import { disableConsoleLog, enableConsoleLog } from "../helper";
+
+beforeAll(disableConsoleLog);
+afterAll(enableConsoleLog);
+
+const roles = {
+   sloppy: {
+      guest: {
+         permissions: [
+            "system.access.admin",
+            "system.schema.read",
+            "system.access.api",
+            "system.config.read",
+            "data.entity.read"
+         ],
+         is_default: true
+      },
+      admin: {
+         is_default: true,
+         implicit_allow: true
+      }
+   },
+   strict: {
+      guest: {
+         permissions: ["system.access.api", "system.config.read", "data.entity.read"],
+         is_default: true
+      },
+      admin: {
+         is_default: true,
+         implicit_allow: true
+      }
+   }
+};
+const configs = {
+   auth: {
+      enabled: true,
+      entity_name: "users",
+      jwt: {
+         secret: secureRandomString(20),
+         issuer: randomString(10)
+      },
+      roles: roles.strict,
+      guard: {
+         enabled: true
+      }
+   },
+   users: {
+      normal: {
+         email: "normal@bknd.io",
+         password: "12345678"
+      },
+      admin: {
+         email: "admin@bknd.io",
+         password: "12345678",
+         role: "admin"
+      }
+   }
+};
+
+function createAuthApp() {
+   const app = createApp({
+      initialConfig: {
+         auth: configs.auth
+      }
+   });
+
+   app.emgr.onEvent(
+      App.Events.AppFirstBoot,
+      async () => {
+         await app.createUser(configs.users.normal);
+         await app.createUser(configs.users.admin);
+      },
+      "sync"
+   );
+
+   return app;
+}
+
+function getCookie(r: Response, name: string) {
+   const cookies = r.headers.get("cookie") ?? r.headers.get("set-cookie");
+   if (!cookies) return;
+   const cookie = cookies.split(";").find((c) => c.trim().startsWith(name));
+   if (!cookie) return;
+   return cookie.split("=")[1];
+}
+
+const fns = <Mode extends "cookie" | "token" = "token">(app: App, mode?: Mode) => {
+   function headers(token?: string, additional?: Record<string, string>) {
+      if (mode === "cookie") {
+         return {
+            cookie: `auth=${token};`,
+            ...additional
+         };
+      }
+
+      return {
+         Authorization: `Bearer ${token}`,
+         "Content-Type": "application/json",
+         ...additional
+      };
+   }
+   function body(obj?: Record<string, any>) {
+      if (mode === "cookie") {
+         const formData = new FormData();
+         for (const key in obj) {
+            formData.append(key, obj[key]);
+         }
+         return formData;
+      }
+
+      return JSON.stringify(obj);
+   }
+
+   return {
+      login: async (
+         user: any
+      ): Promise<{ res: Response; data: Mode extends "token" ? AuthResponse : string }> => {
+         const res = (await app.server.request("/api/auth/password/login", {
+            method: "POST",
+            headers: headers(),
+            body: body(user)
+         })) as Response;
+
+         const data = mode === "cookie" ? getCookie(res, "auth") : await res.json();
+
+         return { res, data };
+      },
+      me: async (token?: string): Promise<Pick<AuthResponse, "user">> => {
+         const res = (await app.server.request("/api/auth/me", {
+            method: "GET",
+            headers: headers(token)
+         })) as Response;
+         return await res.json();
+      }
+   };
+};
+
+describe("integration auth", () => {
+   it("should create users on boot", async () => {
+      const app = createAuthApp();
+      await app.build();
+
+      const { data: users } = await app.em.repository("users").findMany();
+      expect(users.length).toBe(2);
+      expect(users[0].email).toBe(configs.users.normal.email);
+      expect(users[1].email).toBe(configs.users.admin.email);
+   });
+
+   it("should log you in with API", async () => {
+      const app = createAuthApp();
+      await app.build();
+      const $fns = fns(app);
+
+      // login api
+      const { data } = await $fns.login(configs.users.normal);
+      const me = await $fns.me(data.token);
+
+      expect(data.user.email).toBe(me.user.email);
+      expect(me.user.email).toBe(configs.users.normal.email);
+
+      // expect no user with no token
+      expect(await $fns.me()).toEqual({ user: null as any });
+
+      // expect no user with invalid token
+      expect(await $fns.me("invalid")).toEqual({ user: null as any });
+      expect(await $fns.me()).toEqual({ user: null as any });
+   });
+
+   it("should log you in with form and cookie", async () => {
+      const app = createAuthApp();
+      await app.build();
+      const $fns = fns(app, "cookie");
+
+      const { res, data: token } = await $fns.login(configs.users.normal);
+      expect(token).toBeDefined();
+      expect(res.status).toBe(302); // because it redirects
+
+      // test cookie jwt interchangability
+      {
+         // expect token to not work as-is for api endpoints
+         expect(await fns(app).me(token)).toEqual({ user: null as any });
+         // hono adds an additional segment to cookies
+         const apified_token = token.split(".").slice(0, -1).join(".");
+         // now it should work
+         // @todo: maybe add a config to don't allow re-use?
+         expect((await fns(app).me(apified_token)).user.email).toBe(configs.users.normal.email);
+      }
+
+      // test cookie with me endpoint
+      {
+         const me = await $fns.me(token);
+         expect(me.user.email).toBe(configs.users.normal.email);
+
+         // check with invalid & empty
+         expect(await $fns.me("invalid")).toEqual({ user: null as any });
+         expect(await $fns.me()).toEqual({ user: null as any });
+      }
+   });
+
+   it("should check for permissions", async () => {
+      const app = createAuthApp();
+      await app.build();
+
+      await withDisabledConsole(async () => {
+         const res = await app.server.request("/api/system/schema");
+         expect(res.status).toBe(403);
+      });
+   });
+});

app/__test__/media/adapters/StorageCloudinaryAdapter.spec.ts

@@ -3,7 +3,7 @@ import { randomString } from "../../../src/core/utils";
 import { StorageCloudinaryAdapter } from "../../../src/media";
 
 import { config } from "dotenv";
-const dotenvOutput = config({ path: `${import.meta.dir}/../../.env` });
+const dotenvOutput = config({ path: `${import.meta.dir}/../../../.env` });
 const {
    CLOUDINARY_CLOUD_NAME,
    CLOUDINARY_API_KEY,

app/__test__/media/adapters/StorageLocalAdapter.spec.ts

@@ -15,7 +15,7 @@ describe("StorageLocalAdapter", () => {
 
    test("puts an object", async () => {
       objects = (await adapter.listObjects()).length;
-      expect(await adapter.putObject(filename, await file.arrayBuffer())).toBeString();
+      expect(await adapter.putObject(filename, file)).toBeString();
    });
 
    test("lists objects", async () => {

app/__test__/media/adapters/StorageS3Adapter.spec.ts

@@ -3,14 +3,14 @@ import { randomString } from "../../../src/core/utils";
 import { StorageS3Adapter } from "../../../src/media";
 
 import { config } from "dotenv";
-const dotenvOutput = config({ path: `${import.meta.dir}/../../.env` });
+const dotenvOutput = config({ path: `${import.meta.dir}/../../../.env` });
 const { R2_ACCESS_KEY, R2_SECRET_ACCESS_KEY, R2_URL, AWS_ACCESS_KEY, AWS_SECRET_KEY, AWS_S3_URL } =
    dotenvOutput.parsed!;
 
 // @todo: mock r2/s3 responses for faster tests
-const ALL_TESTS = process.env.ALL_TESTS;
+const ALL_TESTS = !!process.env.ALL_TESTS;
 
-describe("Storage", async () => {
+describe.skipIf(ALL_TESTS)("StorageS3Adapter", async () => {
    console.log("ALL_TESTS", process.env.ALL_TESTS);
    const versions = [
       [

app/__test__/media/mime-types.spec.ts

@@ -0,0 +1,37 @@
+import { describe, expect, test } from "bun:test";
+import * as large from "../../src/media/storage/mime-types";
+import * as tiny from "../../src/media/storage/mime-types-tiny";
+
+describe("media/mime-types", () => {
+   test("tiny resolves", () => {
+      const tests = [[".mp4", "video/mp4", ".jpg", "image/jpeg", ".zip", "application/zip"]];
+
+      for (const [ext, mime] of tests) {
+         expect(tiny.guess(ext)).toBe(mime);
+      }
+   });
+
+   test("all tiny resolves to large", () => {
+      for (const [ext, mime] of Object.entries(tiny.M)) {
+         expect(large.guessMimeType("." + ext)).toBe(mime);
+      }
+
+      for (const [type, exts] of Object.entries(tiny.Q)) {
+         for (const ext of exts) {
+            const ex = `${type}/${ext}`;
+            try {
+               expect(large.guessMimeType("." + ext)).toBe(ex);
+            } catch (e) {
+               console.log(`Failed for ${ext}`, {
+                  type,
+                  exts,
+                  ext,
+                  expected: ex,
+                  actual: large.guessMimeType("." + ext)
+               });
+               throw new Error(`Failed for ${ext}`);
+            }
+         }
+      }
+   });
+});

app/__test__/modules/AppAuth.spec.ts

@@ -1,5 +1,7 @@
-import { afterAll, beforeAll, beforeEach, describe, expect, test } from "bun:test";
+import { afterAll, beforeAll, beforeEach, describe, expect, spyOn, test } from "bun:test";
+import { createApp } from "../../src";
 import { AuthController } from "../../src/auth/api/AuthController";
+import { em, entity, make, text } from "../../src/data";
 import { AppAuth, type ModuleBuildContext } from "../../src/modules";
 import { disableConsoleLog, enableConsoleLog } from "../helper";
 import { makeCtx, moduleTestSuite } from "./module-test-suite";
@@ -76,4 +78,87 @@ describe("AppAuth", () => {
          expect(users[0].email).toBe("some@body.com");
       }
    });
+
+   test("registers auth middleware for bknd routes only", async () => {
+      const app = createApp({
+         initialConfig: {
+            auth: {
+               enabled: true,
+               jwt: {
+                  secret: "123456"
+               }
+            }
+         }
+      });
+
+      await app.build();
+      const spy = spyOn(app.module.auth.authenticator, "requestCookieRefresh");
+
+      // register custom route
+      app.server.get("/test", async (c) => c.text("test"));
+
+      // call a system api and then the custom route
+      await app.server.request("/api/system/ping");
+      await app.server.request("/test");
+
+      expect(spy.mock.calls.length).toBe(1);
+   });
+
+   test("should allow additional user fields", async () => {
+      const app = createApp({
+         initialConfig: {
+            auth: {
+               entity_name: "users",
+               enabled: true
+            },
+            data: em({
+               users: entity("users", {
+                  additional: text()
+               })
+            }).toJSON()
+         }
+      });
+
+      await app.build();
+
+      const e = app.modules.em.entity("users");
+      const fields = e.fields.map((f) => f.name);
+      expect(e.type).toBe("system");
+      expect(fields).toContain("additional");
+      expect(fields).toEqual(["id", "additional", "email", "strategy", "strategy_value", "role"]);
+   });
+
+   test("ensure user field configs is always correct", async () => {
+      const app = createApp({
+         initialConfig: {
+            auth: {
+               enabled: true
+            },
+            data: em({
+               users: entity("users", {
+                  strategy: text({
+                     fillable: true,
+                     hidden: false
+                  }),
+                  strategy_value: text({
+                     fillable: true,
+                     hidden: false
+                  })
+               })
+            }).toJSON()
+         }
+      });
+      await app.build();
+
+      const users = app.em.entity("users");
+      const props = ["hidden", "fillable", "required"];
+
+      for (const [name, _authFieldProto] of Object.entries(AppAuth.usersFields)) {
+         const authField = make(name, _authFieldProto as any);
+         const field = users.field(name)!;
+         for (const prop of props) {
+            expect(field.config[prop]).toBe(authField.config[prop]);
+         }
+      }
+   });
 });

app/__test__/modules/AppMedia.spec.ts

@@ -1,7 +1,55 @@
-import { describe } from "bun:test";
+import { describe, expect, test } from "bun:test";
+import { createApp, registries } from "../../src";
+import { em, entity, text } from "../../src/data";
+import { StorageLocalAdapter } from "../../src/media/storage/adapters/StorageLocalAdapter";
 import { AppMedia } from "../../src/modules";
 import { moduleTestSuite } from "./module-test-suite";
 
 describe("AppMedia", () => {
    moduleTestSuite(AppMedia);
+
+   test("should allow additional fields", async () => {
+      registries.media.register("local", StorageLocalAdapter);
+
+      const app = createApp({
+         initialConfig: {
+            media: {
+               entity_name: "media",
+               enabled: true,
+               adapter: {
+                  type: "local",
+                  config: {
+                     path: "./"
+                  }
+               }
+            },
+            data: em({
+               media: entity("media", {
+                  additional: text()
+               })
+            }).toJSON()
+         }
+      });
+
+      await app.build();
+
+      const e = app.modules.em.entity("media");
+      const fields = e.fields.map((f) => f.name);
+      expect(e.type).toBe("system");
+      expect(fields).toContain("additional");
+      expect(fields).toEqual([
+         "id",
+         "additional",
+         "path",
+         "folder",
+         "mime_type",
+         "size",
+         "scope",
+         "etag",
+         "modified_at",
+         "reference",
+         "entity_id",
+         "metadata"
+      ]);
+   });
 });

app/__test__/modules/Module.spec.ts

@@ -0,0 +1,212 @@
+import { describe, expect, test } from "bun:test";
+import { type TSchema, Type, stripMark } from "../../src/core/utils";
+import { EntityManager, em, entity, index, text } from "../../src/data";
+import { DummyConnection } from "../../src/data/connection/DummyConnection";
+import { Module } from "../../src/modules/Module";
+
+function createModule<Schema extends TSchema>(schema: Schema) {
+   class TestModule extends Module<typeof schema> {
+      getSchema() {
+         return schema;
+      }
+      toJSON() {
+         return this.config;
+      }
+      useForceParse() {
+         return true;
+      }
+   }
+
+   return TestModule;
+}
+
+describe("Module", async () => {
+   describe("basic", () => {
+      test("listener", async () => {
+         let result: any;
+
+         const module = createModule(Type.Object({ a: Type.String() }));
+         const m = new module({ a: "test" });
+
+         await m.schema().set({ a: "test2" });
+         m.setListener(async (c) => {
+            await new Promise((r) => setTimeout(r, 10));
+            result = stripMark(c);
+         });
+         await m.schema().set({ a: "test3" });
+         expect(result).toEqual({ a: "test3" });
+      });
+   });
+
+   describe("db schema", () => {
+      class M extends Module {
+         override getSchema() {
+            return Type.Object({});
+         }
+
+         prt = {
+            ensureEntity: this.ensureEntity.bind(this),
+            ensureIndex: this.ensureIndex.bind(this),
+            ensureSchema: this.ensureSchema.bind(this)
+         };
+
+         get em() {
+            return this.ctx.em;
+         }
+      }
+
+      function make(_em: ReturnType<typeof em>) {
+         const em = new EntityManager(
+            Object.values(_em.entities),
+            new DummyConnection(),
+            _em.relations,
+            _em.indices
+         );
+         return new M({} as any, { em, flags: Module.ctx_flags } as any);
+      }
+      function flat(_em: EntityManager) {
+         return {
+            entities: _em.entities.map((e) => ({
+               name: e.name,
+               fields: e.fields.map((f) => f.name),
+               type: e.type
+            })),
+            indices: _em.indices.map((i) => ({
+               name: i.name,
+               entity: i.entity.name,
+               fields: i.fields.map((f) => f.name),
+               unique: i.unique
+            }))
+         };
+      }
+
+      test("no change", () => {
+         const initial = em({});
+
+         const m = make(initial);
+         expect(m.ctx.flags.sync_required).toBe(false);
+
+         expect(flat(make(initial).em)).toEqual({
+            entities: [],
+            indices: []
+         });
+      });
+
+      test("init", () => {
+         const initial = em({
+            users: entity("u", {
+               name: text()
+            })
+         });
+
+         const m = make(initial);
+         expect(m.ctx.flags.sync_required).toBe(false);
+
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name"],
+                  type: "regular"
+               }
+            ],
+            indices: []
+         });
+      });
+
+      test("ensure entity", () => {
+         const initial = em({
+            users: entity("u", {
+               name: text()
+            })
+         });
+
+         const m = make(initial);
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name"],
+                  type: "regular"
+               }
+            ],
+            indices: []
+         });
+
+         // this should add a new entity
+         m.prt.ensureEntity(
+            entity("p", {
+               title: text()
+            })
+         );
+
+         // this should only add the field "important"
+         m.prt.ensureEntity(
+            entity(
+               "u",
+               {
+                  important: text()
+               },
+               undefined,
+               "system"
+            )
+         );
+
+         expect(m.ctx.flags.sync_required).toBe(true);
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name", "important"],
+                  // ensured type must be present
+                  type: "system"
+               },
+               {
+                  name: "p",
+                  fields: ["id", "title"],
+                  type: "regular"
+               }
+            ],
+            indices: []
+         });
+      });
+
+      test("ensure index", () => {
+         const users = entity("u", {
+            name: text(),
+            title: text()
+         });
+         const initial = em({ users }, ({ index }, { users }) => {
+            index(users).on(["title"]);
+         });
+
+         const m = make(initial);
+         m.prt.ensureIndex(index(users).on(["name"]));
+
+         expect(m.ctx.flags.sync_required).toBe(true);
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name", "title"],
+                  type: "regular"
+               }
+            ],
+            indices: [
+               {
+                  name: "idx_u_title",
+                  entity: "u",
+                  fields: ["title"],
+                  unique: false
+               },
+               {
+                  name: "idx_u_name",
+                  entity: "u",
+                  fields: ["name"],
+                  unique: false
+               }
+            ]
+         });
+      });
+   });
+});

app/__test__/modules/ModuleManager.spec.ts

@@ -1,9 +1,9 @@
 import { describe, expect, test } from "bun:test";
-import { mark, stripMark } from "../src/core/utils";
+import { stripMark } from "../../src/core/utils";
-import { entity, text } from "../src/data";
+import { entity, text } from "../../src/data";
-import { ModuleManager, getDefaultConfig } from "../src/modules/ModuleManager";
+import { ModuleManager, getDefaultConfig } from "../../src/modules/ModuleManager";
-import { CURRENT_VERSION, TABLE_NAME } from "../src/modules/migrations";
+import { CURRENT_VERSION, TABLE_NAME } from "../../src/modules/migrations";
-import { getDummyConnection } from "./helper";
+import { getDummyConnection } from "../helper";
 
 describe("ModuleManager", async () => {
    test("s1: no config, no build", async () => {

app/__test__/modules/module-test-suite.ts

@@ -2,10 +2,11 @@ import { beforeEach, describe, expect, it } from "bun:test";
 
 import { Hono } from "hono";
 import { Guard } from "../../src/auth";
+import { DebugLogger } from "../../src/core";
 import { EventManager } from "../../src/core/events";
 import { Default, stripMark } from "../../src/core/utils";
 import { EntityManager } from "../../src/data";
-import type { Module, ModuleBuildContext } from "../../src/modules/Module";
+import { Module, type ModuleBuildContext } from "../../src/modules/Module";
 import { getDummyConnection } from "../helper";
 
 export function makeCtx(overrides?: Partial<ModuleBuildContext>): ModuleBuildContext {
@@ -16,6 +17,8 @@ export function makeCtx(overrides?: Partial<ModuleBuildContext>): ModuleBuildCon
       em: new EntityManager([], dummyConnection),
       emgr: new EventManager(),
       guard: new Guard(),
+      flags: Module.ctx_flags,
+      logger: new DebugLogger(false),
       ...overrides
    };
 }

app/build.ts

@@ -1,82 +1,49 @@
 import { $ } from "bun";
-import * as esbuild from "esbuild";
-import postcss from "esbuild-postcss";
 import * as tsup from "tsup";
-import { guessMimeType } from "./src/media/storage/mime-types";
 
 const args = process.argv.slice(2);
 const watch = args.includes("--watch");
 const minify = args.includes("--minify");
 const types = args.includes("--types");
 const sourcemap = args.includes("--sourcemap");
+const clean = args.includes("--clean");
+
+if (clean) {
+   console.log("Cleaning dist (w/o static)");
+   await $`find dist -mindepth 1 ! -path "dist/static/*" ! -path "dist/static" -exec rm -rf {} +`;
+}
+
+let types_running = false;
+function buildTypes() {
+   if (types_running || !types) return;
+   types_running = true;
 
-await $`rm -rf dist`;
-if (types) {
    Bun.spawn(["bun", "build:types"], {
+      stdout: "inherit",
       onExit: () => {
          console.log("Types built");
+         Bun.spawn(["bun", "tsc-alias"], {
+            stdout: "inherit",
+            onExit: () => {
+               console.log("Types aliased");
+               types_running = false;
+            }
+         });
       }
    });
 }
 
-/**
+let watcher_timeout: any;
- * Build static assets
+function delayTypes() {
- * Using esbuild because tsup doesn't include "react"
+   if (!watch || !types) return;
- */
+   if (watcher_timeout) {
-const result = await esbuild.build({
+      clearTimeout(watcher_timeout);
-   minify,
-   sourcemap,
-   entryPoints: ["src/ui/main.tsx"],
-   entryNames: "[dir]/[name]-[hash]",
-   outdir: "dist/static",
-   platform: "browser",
-   bundle: true,
-   splitting: true,
-   metafile: true,
-   drop: ["console", "debugger"],
-   inject: ["src/ui/inject.js"],
-   target: "es2022",
-   format: "esm",
-   plugins: [postcss()],
-   loader: {
-      ".svg": "dataurl",
-      ".js": "jsx"
-   },
-   define: {
-      __isDev: "0",
-      "process.env.NODE_ENV": '"production"'
-   },
-   chunkNames: "chunks/[name]-[hash]"
-});
-
-// Write manifest
-{
-   const manifest: Record<string, object> = {};
-   const toAsset = (output: string) => {
-      const name = output.split("/").pop()!;
-      return {
-         name,
-         path: output,
-         mime: guessMimeType(name)
-      };
-   };
-
-   const info = Object.entries(result.metafile.outputs)
-      .filter(([, meta]) => {
-         return meta.entryPoint && meta.entryPoint === "src/ui/main.tsx";
-      })
-      .map(([output, meta]) => ({ output, meta }));
-
-   for (const { output, meta } of info) {
-      manifest[meta.entryPoint as string] = toAsset(output);
-      if (meta.cssBundle) {
-         manifest["src/ui/main.css"] = toAsset(meta.cssBundle);
-      }
    }
+   watcher_timeout = setTimeout(buildTypes, 1000);
+}
 
-   const manifest_file = "dist/static/manifest.json";
+if (types && !watch) {
-   await Bun.write(manifest_file, JSON.stringify(manifest, null, 2));
+   buildTypes();
-   console.log(`Manifest written to ${manifest_file}`, manifest);
 }
 
 /**
@@ -91,11 +58,14 @@ await tsup.build({
    external: ["bun:test", "@libsql/client"],
    metafile: true,
    platform: "browser",
-   format: ["esm", "cjs"],
+   format: ["esm"],
    splitting: false,
    treeshake: true,
    loader: {
       ".svg": "dataurl"
+   },
+   onSuccess: async () => {
+      delayTypes();
    }
 });
 
@@ -106,22 +76,77 @@ await tsup.build({
    minify,
    sourcemap,
    watch,
-   entry: ["src/ui/index.ts", "src/ui/client/index.ts", "src/ui/main.css"],
+   entry: ["src/ui/index.ts", "src/ui/client/index.ts", "src/ui/main.css", "src/ui/styles.css"],
    outDir: "dist/ui",
-   external: ["bun:test", "react", "react-dom", "use-sync-external-store"],
+   external: [
+      "bun:test",
+      "react",
+      "react-dom",
+      "react/jsx-runtime",
+      "react/jsx-dev-runtime",
+      "use-sync-external-store",
+      /codemirror/,
+      "@xyflow/react",
+      "@mantine/core"
+   ],
    metafile: true,
    platform: "browser",
-   format: ["esm", "cjs"],
+   format: ["esm"],
-   splitting: true,
+   splitting: false,
+   bundle: true,
    treeshake: true,
    loader: {
       ".svg": "dataurl"
    },
+   esbuildOptions: (options) => {
+      options.logLevel = "silent";
+   },
    onSuccess: async () => {
-      console.log("--- ui built");
+      delayTypes();
+   }
+});
+
+/**
+ * Building UI Elements
+ * - tailwind-merge is mocked, no exclude
+ * - ui/client is external, and after built replaced with "bknd/client"
+ */
+await tsup.build({
+   minify,
+   sourcemap,
+   watch,
+   entry: ["src/ui/elements/index.ts"],
+   outDir: "dist/ui/elements",
+   external: [
+      "ui/client",
+      "react",
+      "react-dom",
+      "react/jsx-runtime",
+      "react/jsx-dev-runtime",
+      "use-sync-external-store"
+   ],
+   metafile: true,
+   platform: "browser",
+   format: ["esm"],
+   splitting: false,
+   bundle: true,
+   treeshake: true,
+   loader: {
+      ".svg": "dataurl"
    },
    esbuildOptions: (options) => {
-      options.chunkNames = "chunks/[name]-[hash]";
+      options.alias = {
+         // not important for elements, mock to reduce bundle
+         "tailwind-merge": "./src/ui/elements/mocks/tailwind-merge.ts"
+      };
+   },
+   onSuccess: async () => {
+      // manually replace ui/client with bknd/client
+      const path = "./dist/ui/elements/index.js";
+      const bundle = await Bun.file(path).text();
+      await Bun.write(path, bundle.replaceAll("ui/client", "bknd/client"));
+
+      delayTypes();
    }
 });
 
@@ -133,7 +158,7 @@ function baseConfig(adapter: string): tsup.Options {
       minify,
       sourcemap,
       watch,
-      entry: [`src/adapter/${adapter}`],
+      entry: [`src/adapter/${adapter}/index.ts`],
       format: ["esm"],
       platform: "neutral",
       outDir: `dist/adapter/${adapter}`,
@@ -148,41 +173,29 @@ function baseConfig(adapter: string): tsup.Options {
       ],
       metafile: true,
       splitting: false,
-      treeshake: true
+      treeshake: true,
+      onSuccess: async () => {
+         delayTypes();
+      }
    };
 }
 
+await tsup.build(baseConfig("remix"));
+await tsup.build(baseConfig("bun"));
+await tsup.build(baseConfig("astro"));
+await tsup.build(baseConfig("cloudflare"));
+
 await tsup.build({
    ...baseConfig("vite"),
    platform: "node"
 });
 
-await tsup.build({
-   ...baseConfig("cloudflare")
-});
-
 await tsup.build({
    ...baseConfig("nextjs"),
-   format: ["esm", "cjs"],
    platform: "node"
 });
 
-await tsup.build({
-   ...baseConfig("remix"),
-   format: ["esm", "cjs"]
-});
-
-await tsup.build({
-   ...baseConfig("bun")
-});
-
 await tsup.build({
    ...baseConfig("node"),
-   platform: "node",
+   platform: "node"
-   format: ["esm", "cjs"]
-});
-
-await tsup.build({
-   ...baseConfig("astro"),
-   format: ["esm", "cjs"]
 });

app/bunfig.toml

@@ -1,2 +1,5 @@
 [install]
-registry = "http://localhost:4873"
+#registry = "http://localhost:4873"
+
+[test]
+coverageSkipTestFiles = true

app/package.json

@@ -3,66 +3,74 @@
   "type": "module",
   "sideEffects": false,
   "bin": "./dist/cli/index.js",
-  "version": "0.3.2",
+  "version": "0.6.2",
+  "description": "Lightweight Firebase/Supabase alternative built to run anywhere — incl. Next.js, Remix, Astro, Cloudflare, Bun, Node, AWS Lambda & more.",
+  "homepage": "https://bknd.io",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/bknd-io/bknd.git"
+  },
+  "bugs": {
+    "url": "https://github.com/bknd-io/bknd/issues"
+  },
   "scripts": {
-    "build:all": "bun run build && bun run build:cli",
     "dev": "vite",
     "test": "ALL_TESTS=1 bun test --bail",
+    "test:coverage": "ALL_TESTS=1 bun test --bail --coverage",
     "build": "NODE_ENV=production bun run build.ts --minify --types",
+    "build:all": "rm -rf dist && bun run build:static && NODE_ENV=production bun run build.ts --minify --types --clean && bun run build:cli",
+    "build:cli": "bun build src/cli/index.ts --target node --outdir dist/cli --minify",
+    "build:static": "vite build",
     "watch": "bun run build.ts --types --watch",
     "types": "bun tsc --noEmit",
     "clean:types": "find ./dist -name '*.d.ts' -delete && rm -f ./dist/tsconfig.tsbuildinfo",
-    "build:types": "tsc --emitDeclarationOnly",
+    "build:types": "tsc --emitDeclarationOnly && tsc-alias",
-    "build:css": "bun tailwindcss -i src/ui/main.css -o ./dist/static/styles.css",
-    "watch:css": "bun tailwindcss --watch -i src/ui/main.css -o ./dist/styles.css",
     "updater": "bun x npm-check-updates -ui",
-    "build:cli": "bun build src/cli/index.ts --target node --outdir dist/cli --minify",
     "cli": "LOCAL=1 bun src/cli/index.ts",
-    "prepublishOnly": "bun run build:all"
+    "prepublishOnly": "bun run types && bun run test && bun run build:all && cp ../README.md ./",
+    "postpublish": "rm -f README.md"
   },
   "license": "FSL-1.1-MIT",
   "dependencies": {
+    "@cfworker/json-schema": "^2.0.1",
     "@libsql/client": "^0.14.0",
-    "@tanstack/react-form": "0.19.2",
     "@sinclair/typebox": "^0.32.34",
+    "@tanstack/react-form": "0.19.2",
+    "aws4fetch": "^1.0.18",
+    "dayjs": "^1.11.13",
+    "fast-xml-parser": "^4.4.0",
+    "hono": "^4.6.12",
     "kysely": "^0.27.4",
     "liquidjs": "^10.15.0",
     "lodash-es": "^4.17.21",
-    "hono": "^4.6.12",
-    "fast-xml-parser": "^4.4.0",
-    "@cfworker/json-schema": "^2.0.1",
-    "dayjs": "^1.11.13",
     "oauth4webapi": "^2.11.1",
-    "aws4fetch": "^1.0.18"
+    "swr": "^2.2.5",
-  },
+    "json-schema-form-react": "^0.0.2",
-  "devDependencies": {
+    "@uiw/react-codemirror": "^4.23.6",
-    "@aws-sdk/client-s3": "^3.613.0",
     "@codemirror/lang-html": "^6.4.9",
     "@codemirror/lang-json": "^6.0.1",
     "@codemirror/lang-liquid": "^6.2.1",
+    "@xyflow/react": "^12.3.2",
+    "@mantine/core": "^7.13.4",
+    "@hello-pangea/dnd": "^17.0.0"
+  },
+  "devDependencies": {
+    "@aws-sdk/client-s3": "^3.613.0",
     "@dagrejs/dagre": "^1.1.4",
-    "@hello-pangea/dnd": "^17.0.0",
     "@hono/typebox-validator": "^0.2.6",
     "@hono/vite-dev-server": "^0.17.0",
     "@hono/zod-validator": "^0.4.1",
     "@hookform/resolvers": "^3.9.1",
     "@libsql/kysely-libsql": "^0.4.1",
-    "@mantine/core": "^7.13.4",
-    "@mantine/hooks": "^7.13.4",
-    "@mantine/modals": "^7.13.4",
-    "@mantine/notifications": "^7.13.5",
     "@radix-ui/react-scroll-area": "^1.2.0",
     "@rjsf/core": "^5.22.2",
     "@tabler/icons-react": "3.18.0",
-    "@tanstack/react-query": "^5.59.16",
-    "@tanstack/react-query-devtools": "^5.59.16",
     "@types/node": "^22.10.0",
     "@types/react": "^18.3.12",
     "@types/react-dom": "^18.3.1",
-    "@uiw/react-codemirror": "^4.23.6",
     "@vitejs/plugin-react": "^4.3.3",
-    "@xyflow/react": "^12.3.2",
     "autoprefixer": "^10.4.20",
+    "clsx": "^2.1.1",
     "esbuild-postcss": "^0.0.4",
     "jotai": "^2.10.1",
     "open": "^10.1.0",
@@ -73,9 +81,11 @@
     "react-hook-form": "^7.53.1",
     "react-icons": "5.2.1",
     "react-json-view-lite": "^2.0.1",
+    "sql-formatter": "^15.4.9",
     "tailwind-merge": "^2.5.4",
     "tailwindcss": "^3.4.14",
     "tailwindcss-animate": "^1.0.7",
+    "tsc-alias": "^1.8.10",
     "tsup": "^8.3.5",
     "vite": "^5.4.10",
     "vite-plugin-static-copy": "^2.0.0",
@@ -91,80 +101,86 @@
   },
   "main": "./dist/index.js",
   "module": "./dist/index.js",
-  "types": "./dist/index.d.ts",
+  "types": "./dist/types/index.d.ts",
   "exports": {
     ".": {
-      "types": "./dist/index.d.ts",
+      "types": "./dist/types/index.d.ts",
       "import": "./dist/index.js",
       "require": "./dist/index.cjs"
     },
     "./ui": {
-      "types": "./dist/ui/index.d.ts",
+      "types": "./dist/types/ui/index.d.ts",
       "import": "./dist/ui/index.js",
       "require": "./dist/ui/index.cjs"
     },
+    "./elements": {
+      "types": "./dist/types/ui/elements/index.d.ts",
+      "import": "./dist/ui/elements/index.js",
+      "require": "./dist/ui/elements/index.cjs"
+    },
     "./client": {
-      "types": "./dist/ui/client/index.d.ts",
+      "types": "./dist/types/ui/client/index.d.ts",
       "import": "./dist/ui/client/index.js",
       "require": "./dist/ui/client/index.cjs"
     },
     "./data": {
-      "types": "./dist/data/index.d.ts",
+      "types": "./dist/types/data/index.d.ts",
       "import": "./dist/data/index.js",
       "require": "./dist/data/index.cjs"
     },
     "./core": {
-      "types": "./dist/core/index.d.ts",
+      "types": "./dist/types/core/index.d.ts",
       "import": "./dist/core/index.js",
       "require": "./dist/core/index.cjs"
     },
     "./utils": {
-      "types": "./dist/core/utils/index.d.ts",
+      "types": "./dist/types/core/utils/index.d.ts",
       "import": "./dist/core/utils/index.js",
       "require": "./dist/core/utils/index.cjs"
     },
     "./cli": {
-      "types": "./dist/cli/index.d.ts",
+      "types": "./dist/types/cli/index.d.ts",
       "import": "./dist/cli/index.js",
       "require": "./dist/cli/index.cjs"
     },
     "./adapter/cloudflare": {
-      "types": "./dist/adapter/cloudflare/index.d.ts",
+      "types": "./dist/types/adapter/cloudflare/index.d.ts",
       "import": "./dist/adapter/cloudflare/index.js",
       "require": "./dist/adapter/cloudflare/index.cjs"
     },
     "./adapter/vite": {
-      "types": "./dist/adapter/vite/index.d.ts",
+      "types": "./dist/types/adapter/vite/index.d.ts",
       "import": "./dist/adapter/vite/index.js",
       "require": "./dist/adapter/vite/index.cjs"
     },
     "./adapter/nextjs": {
-      "types": "./dist/adapter/nextjs/index.d.ts",
+      "types": "./dist/types/adapter/nextjs/index.d.ts",
       "import": "./dist/adapter/nextjs/index.js",
       "require": "./dist/adapter/nextjs/index.cjs"
     },
     "./adapter/remix": {
-      "types": "./dist/adapter/remix/index.d.ts",
+      "types": "./dist/types/adapter/remix/index.d.ts",
       "import": "./dist/adapter/remix/index.js",
       "require": "./dist/adapter/remix/index.cjs"
     },
     "./adapter/bun": {
-      "types": "./dist/adapter/bun/index.d.ts",
+      "types": "./dist/types/adapter/bun/index.d.ts",
       "import": "./dist/adapter/bun/index.js",
       "require": "./dist/adapter/bun/index.cjs"
     },
     "./adapter/node": {
-      "types": "./dist/adapter/node/index.d.ts",
+      "types": "./dist/types/adapter/node/index.d.ts",
       "import": "./dist/adapter/node/index.js",
       "require": "./dist/adapter/node/index.cjs"
     },
     "./adapter/astro": {
-      "types": "./dist/adapter/astro/index.d.ts",
+      "types": "./dist/types/adapter/astro/index.d.ts",
       "import": "./dist/adapter/astro/index.js",
       "require": "./dist/adapter/astro/index.cjs"
     },
-    "./dist/styles.css": "./dist/ui/main.css",
+    "./dist/main.css": "./dist/ui/main.css",
-    "./dist/manifest.json": "./dist/static/manifest.json"
+    "./dist/styles.css": "./dist/ui/styles.css",
+    "./dist/manifest.json": "./dist/static/.vite/manifest.json"
   },
   "publishConfig": {
     "access": "public"
@@ -177,5 +193,21 @@
     "!dist/**/*.map",
     "!dist/metafile*",
     "!dist/**/metafile*"
+  ],
+  "keywords": [
+    "backend",
+    "database",
+    "authentication",
+    "media",
+    "workflows",
+    "api",
+    "jwt",
+    "serverless",
+    "cloudflare",
+    "nextjs",
+    "remix",
+    "astro",
+    "bun",
+    "node"
   ]
 }

app/src/Api.ts

@@ -1,3 +1,4 @@
+import type { SafeUser } from "auth";
 import { AuthApi } from "auth/api/AuthApi";
 import { DataApi } from "data/api/DataApi";
 import { decode } from "hono/jwt";
@@ -5,7 +6,7 @@ import { omit } from "lodash-es";
 import { MediaApi } from "media/api/MediaApi";
 import { SystemApi } from "modules/SystemApi";
 
-export type TApiUser = object;
+export type TApiUser = SafeUser;
 
 declare global {
    interface Window {
@@ -24,6 +25,12 @@ export type ApiOptions = {
    localStorage?: boolean;
 };
 
+export type AuthState = {
+   token?: string;
+   user?: TApiUser;
+   verified: boolean;
+};
+
 export class Api {
    private token?: string;
    private user?: TApiUser;
@@ -50,6 +57,10 @@ export class Api {
       this.buildApis();
    }
 
+   get baseUrl() {
+      return this.options.host;
+   }
+
    get tokenKey() {
       return this.options.key ?? "auth";
    }
@@ -85,7 +96,11 @@ export class Api {
 
    updateToken(token?: string, rebuild?: boolean) {
       this.token = token;
-      this.user = token ? omit(decode(token).payload as any, ["iat", "iss", "exp"]) : undefined;
+      if (token) {
+         this.user = omit(decode(token).payload as any, ["iat", "iss", "exp"]) as any;
+      } else {
+         this.user = undefined;
+      }
 
       if (this.options.localStorage) {
          const key = this.tokenKey;
@@ -105,7 +120,7 @@ export class Api {
       return this;
    }
 
-   getAuthState() {
+   getAuthState(): AuthState {
       return {
          token: this.token,
          user: this.user,
@@ -113,6 +128,30 @@ export class Api {
       };
    }
 
+   async getVerifiedAuthState(): Promise<AuthState> {
+      await this.verifyAuth();
+      return this.getAuthState();
+   }
+
+   async verifyAuth() {
+      if (!this.token) {
+         this.markAuthVerified(false);
+         return;
+      }
+
+      try {
+         const res = await this.auth.me();
+         if (!res.ok || !res.body.user) {
+            throw new Error();
+         }
+
+         this.markAuthVerified(true);
+      } catch (e) {
+         this.markAuthVerified(false);
+         this.updateToken(undefined);
+      }
+   }
+
    getUser(): TApiUser | null {
       return this.user || null;
    }

app/src/App.ts

@@ -1,3 +1,4 @@
+import type { CreateUserPayload } from "auth/AppAuth";
 import { Event } from "core/events";
 import { Connection, type LibSqlCredentials, LibsqlConnection } from "data";
 import {
@@ -10,15 +11,19 @@ import * as SystemPermissions from "modules/permissions";
 import { AdminController, type AdminControllerOptions } from "modules/server/AdminController";
 import { SystemController } from "modules/server/SystemController";
 
-export type AppPlugin<DB> = (app: App<DB>) => void;
+export type AppPlugin = (app: App) => Promise<void> | void;
 
-export class AppConfigUpdatedEvent extends Event<{ app: App }> {
+abstract class AppEvent<A = {}> extends Event<{ app: App } & A> {}
+export class AppConfigUpdatedEvent extends AppEvent {
    static override slug = "app-config-updated";
 }
-export class AppBuiltEvent extends Event<{ app: App }> {
+export class AppBuiltEvent extends AppEvent {
    static override slug = "app-built";
 }
-export const AppEvents = { AppConfigUpdatedEvent, AppBuiltEvent } as const;
+export class AppFirstBoot extends AppEvent {
+   static override slug = "app-first-boot";
+}
+export const AppEvents = { AppConfigUpdatedEvent, AppBuiltEvent, AppFirstBoot } as const;
 
 export type CreateAppConfig = {
    connection?:
@@ -28,29 +33,48 @@ export type CreateAppConfig = {
            config: LibSqlCredentials;
         };
    initialConfig?: InitialModuleConfigs;
-   plugins?: AppPlugin<any>[];
+   plugins?: AppPlugin[];
    options?: Omit<ModuleManagerOptions, "initial" | "onUpdated">;
 };
 
 export type AppConfig = InitialModuleConfigs;
 
-export class App<DB = any> {
+export class App {
    modules: ModuleManager;
    static readonly Events = AppEvents;
+   adminController?: AdminController;
+   private trigger_first_boot = false;
 
    constructor(
       private connection: Connection,
       _initialConfig?: InitialModuleConfigs,
-      private plugins: AppPlugin<DB>[] = [],
+      private plugins: AppPlugin[] = [],
       moduleManagerOptions?: ModuleManagerOptions
    ) {
       this.modules = new ModuleManager(connection, {
          ...moduleManagerOptions,
          initial: _initialConfig,
          onUpdated: async (key, config) => {
-            //console.log("[APP] config updated", key, config);
+            // if the EventManager was disabled, we assume we shouldn't
+            // respond to events, such as "onUpdated".
+            if (!this.emgr.enabled) {
+               console.warn("[APP] config updated, but event manager is disabled, skip.");
+               return;
+            }
+
+            console.log("[APP] config updated", key);
             await this.build({ sync: true, save: true });
             await this.emgr.emit(new AppConfigUpdatedEvent({ app: this }));
+         },
+         onFirstBoot: async () => {
+            console.log("[APP] first boot");
+            this.trigger_first_boot = true;
+         },
+         onServerInit: async (server) => {
+            server.use(async (c, next) => {
+               c.set("app", this);
+               await next();
+            });
          }
       });
       this.modules.ctx().emgr.registerEvents(AppEvents);
@@ -70,32 +94,46 @@ export class App<DB = any> {
          //console.log("syncing", syncResult);
       }
 
+      const { guard, server } = this.modules.ctx();
+
       // load system controller
-      this.modules.ctx().guard.registerPermissions(Object.values(SystemPermissions));
+      guard.registerPermissions(Object.values(SystemPermissions));
-      this.modules.server.route("/api/system", new SystemController(this).getController());
+      server.route("/api/system", new SystemController(this).getController());
 
       // load plugins
       if (this.plugins.length > 0) {
-         this.plugins.forEach((plugin) => plugin(this));
+         await Promise.all(this.plugins.map((plugin) => plugin(this)));
       }
 
-      //console.log("emitting built", options);
       await this.emgr.emit(new AppBuiltEvent({ app: this }));
 
-      // not found on any not registered api route
+      server.all("/api/*", async (c) => c.notFound());
-      this.modules.server.all("/api/*", async (c) => c.notFound());
 
       if (options?.save) {
          await this.modules.save();
       }
+
+      // first boot is set from ModuleManager when there wasn't a config table
+      if (this.trigger_first_boot) {
+         this.trigger_first_boot = false;
+         await this.emgr.emit(new AppFirstBoot({ app: this }));
+      }
    }
 
    mutateConfig<Module extends keyof Modules>(module: Module) {
       return this.modules.get(module).schema();
    }
 
+   get server() {
+      return this.modules.server;
+   }
+
+   get em() {
+      return this.modules.ctx().em;
+   }
+
    get fetch(): any {
-      return this.modules.server.fetch;
+      return this.server.fetch;
    }
 
    get module() {
@@ -119,7 +157,8 @@ export class App<DB = any> {
 
    registerAdminController(config?: AdminControllerOptions) {
       // register admin
-      this.modules.server.route("/", new AdminController(this, config).getController());
+      this.adminController = new AdminController(this, config);
+      this.modules.server.route(config?.basepath ?? "/", this.adminController.getController());
       return this;
    }
 
@@ -130,6 +169,10 @@ export class App<DB = any> {
    static create(config: CreateAppConfig) {
       return createApp(config);
    }
+
+   async createUser(p: CreateUserPayload) {
+      return this.module.auth.createUser(p);
+   }
 }
 
 export function createApp(config: CreateAppConfig = {}) {

app/src/adapter/astro/astro.adapter.ts

@@ -1,4 +1,7 @@
-import { Api, type ApiOptions, App, type CreateAppConfig } from "bknd";
+import { type FrameworkBkndConfig, createFrameworkApp } from "adapter";
+import { Api, type ApiOptions, type App } from "bknd";
+
+export type AstroBkndConfig = FrameworkBkndConfig;
 
 type TAstro = {
    request: Request;
@@ -18,12 +21,10 @@ export function getApi(Astro: TAstro, options: Options = { mode: "static" }) {
 }
 
 let app: App;
-export function serve(config: CreateAppConfig) {
+export function serve(config: AstroBkndConfig = {}) {
    return async (args: TAstro) => {
       if (!app) {
-         app = App.create(config);
+         app = await createFrameworkApp(config);
-
-         await app.build();
       }
       return app.fetch(args.request);
    };

app/src/adapter/bun/bun.adapter.ts

@@ -1,56 +1,54 @@
 /// <reference types="bun-types" />
 
 import path from "node:path";
-import { App, type CreateAppConfig } from "bknd";
+import type { App } from "bknd";
-import type { Serve, ServeOptions } from "bun";
+import type { ServeOptions } from "bun";
+import { config } from "core";
 import { serveStatic } from "hono/bun";
+import { type RuntimeBkndConfig, createRuntimeApp } from "../index";
 
 let app: App;
-export async function createApp(_config: Partial<CreateAppConfig> = {}, distPath?: string) {
+
+export type BunBkndConfig = RuntimeBkndConfig & Omit<ServeOptions, "fetch">;
+
+export async function createApp({ distPath, ...config }: RuntimeBkndConfig = {}) {
    const root = path.resolve(distPath ?? "./node_modules/bknd/dist", "static");
 
    if (!app) {
-      app = App.create(_config);
+      app = await createRuntimeApp({
-
+         ...config,
-      app.emgr.on(
+         registerLocalMedia: true,
-         "app-built",
+         serveStatic: serveStatic({ root })
-         async () => {
+      });
-            app.modules.server.get(
-               "/*",
-               serveStatic({
-                  root
-               })
-            );
-            app.registerAdminController();
-         },
-         "sync"
-      );
-
-      await app.build();
    }
 
    return app;
 }
 
-export type BunAdapterOptions = Omit<ServeOptions, "fetch"> &
-   CreateAppConfig & {
-      distPath?: string;
-   };
-
 export function serve({
    distPath,
    connection,
    initialConfig,
    plugins,
    options,
-   port = 1337,
+   port = config.server.default_port,
+   onBuilt,
+   buildConfig,
    ...serveOptions
-}: BunAdapterOptions = {}) {
+}: BunBkndConfig = {}) {
    Bun.serve({
       ...serveOptions,
       port,
       fetch: async (request: Request) => {
-         const app = await createApp({ connection, initialConfig, plugins, options }, distPath);
+         const app = await createApp({
+            connection,
+            initialConfig,
+            plugins,
+            options,
+            onBuilt,
+            buildConfig,
+            distPath
+         });
          return app.fetch(request);
       }
    });

app/src/adapter/cloudflare/cloudflare-workers.adapter.ts

@@ -1,21 +1,37 @@
-import { DurableObject } from "cloudflare:workers";
+import type { CreateAppConfig } from "bknd";
-import { App, type CreateAppConfig } from "bknd";
 import { Hono } from "hono";
 import { serveStatic } from "hono/cloudflare-workers";
-import type { BkndConfig, CfBkndModeCache } from "../index";
+import type { FrameworkBkndConfig } from "../index";
+import { getCached } from "./modes/cached";
+import { getDurable } from "./modes/durable";
+import { getFresh, getWarm } from "./modes/fresh";
 
-type Context = {
+export type CloudflareBkndConfig<Env = any> = Omit<FrameworkBkndConfig, "app"> & {
-   request: Request;
+   app: CreateAppConfig | ((env: Env) => CreateAppConfig);
-   env: any;
+   mode?: "warm" | "fresh" | "cache" | "durable";
-   ctx: ExecutionContext;
+   bindings?: (env: Env) => {
-   manifest: any;
+      kv?: KVNamespace;
+      dobj?: DurableObjectNamespace;
+   };
+   key?: string;
+   keepAliveSeconds?: number;
+   forceHttps?: boolean;
+   manifest?: string;
+   setAdminHtml?: boolean;
    html?: string;
 };
 
-export function serve(_config: BkndConfig, manifest?: string, html?: string) {
+export type Context = {
+   request: Request;
+   env: any;
+   ctx: ExecutionContext;
+};
+
+export function serve(config: CloudflareBkndConfig) {
    return {
       async fetch(request: Request, env: any, ctx: ExecutionContext) {
          const url = new URL(request.url);
+         const manifest = config.manifest;
 
          if (manifest) {
             const pathname = url.pathname.slice(1);
@@ -26,13 +42,10 @@ export function serve(_config: BkndConfig, manifest?: string, html?: string) {
                hono.all("*", async (c, next) => {
                   const res = await serveStatic({
                      path: `./${pathname}`,
-                     manifest,
+                     manifest
-                     onNotFound: (path) => console.log("not found", path)
                   })(c as any, next);
                   if (res instanceof Response) {
-                     const ttl = pathname.startsWith("assets/")
+                     const ttl = 60 * 60 * 24 * 365;
-                        ? 60 * 60 * 24 * 365 // 1 year
-                        : 60 * 5; // 5 minutes
                      res.headers.set("Cache-Control", `public, max-age=${ttl}`);
                      return res;
                   }
@@ -44,218 +57,23 @@ export function serve(_config: BkndConfig, manifest?: string, html?: string) {
             }
          }
 
-         const config = {
+         config.setAdminHtml = config.setAdminHtml && !!config.manifest;
-            ..._config,
-            setAdminHtml: _config.setAdminHtml ?? !!manifest
-         };
-         const context = { request, env, ctx, manifest, html };
-         const mode = config.cloudflare?.mode?.(env);
 
-         if (!mode) {
+         const context = { request, env, ctx } as Context;
-            console.log("serving fresh...");
+         const mode = config.mode ?? "warm";
-            const app = await getFresh(config, context);
-            return app.fetch(request, env);
-         } else if ("cache" in mode) {
-            console.log("serving cached...");
-            const app = await getCached(config as any, context);
-            return app.fetch(request, env);
-         } else if ("durableObject" in mode) {
-            console.log("serving durable...");
 
-            if (config.onBuilt) {
+         switch (mode) {
-               console.log("onBuilt() is not supported with DurableObject mode");
+            case "fresh":
-            }
+               return await getFresh(config, context);
-
+            case "warm":
-            const start = performance.now();
+               return await getWarm(config, context);
-
+            case "cache":
-            const durable = mode.durableObject;
+               return await getCached(config, context);
-            const id = durable.idFromName(mode.key);
+            case "durable":
-            const stub = durable.get(id) as unknown as DurableBkndApp;
+               return await getDurable(config, context);
-
+            default:
-            const create_config = typeof config.app === "function" ? config.app(env) : config.app;
+               throw new Error(`Unknown mode ${mode}`);
-
-            const res = await stub.fire(request, {
-               config: create_config,
-               html,
-               keepAliveSeconds: mode.keepAliveSeconds,
-               setAdminHtml: config.setAdminHtml
-            });
-
-            const headers = new Headers(res.headers);
-            headers.set("X-TTDO", String(performance.now() - start));
-
-            return new Response(res.body, {
-               status: res.status,
-               statusText: res.statusText,
-               headers
-            });
          }
       }
    };
 }
-
-async function getFresh(config: BkndConfig, { env, html }: Context) {
-   const create_config = typeof config.app === "function" ? config.app(env) : config.app;
-   const app = App.create(create_config);
-
-   if (config.onBuilt) {
-      app.emgr.onEvent(
-         App.Events.AppBuiltEvent,
-         async ({ params: { app } }) => {
-            config.onBuilt!(app);
-         },
-         "sync"
-      );
-   }
-   await app.build();
-
-   if (config.setAdminHtml) {
-      app.registerAdminController({ html });
-   }
-
-   return app;
-}
-
-async function getCached(
-   config: BkndConfig & { cloudflare: { mode: CfBkndModeCache } },
-   { env, html, ctx }: Context
-) {
-   const { cache, key } = config.cloudflare.mode(env) as ReturnType<CfBkndModeCache>;
-   const create_config = typeof config.app === "function" ? config.app(env) : config.app;
-
-   const cachedConfig = await cache.get(key);
-   const initialConfig = cachedConfig ? JSON.parse(cachedConfig) : undefined;
-
-   const app = App.create({ ...create_config, initialConfig });
-
-   async function saveConfig(__config: any) {
-      ctx.waitUntil(cache.put(key, JSON.stringify(__config)));
-   }
-
-   if (config.onBuilt) {
-      app.emgr.onEvent(
-         App.Events.AppBuiltEvent,
-         async ({ params: { app } }) => {
-            app.module.server.client.get("/__bknd/cache", async (c) => {
-               await cache.delete(key);
-               return c.json({ message: "Cache cleared" });
-            });
-            app.registerAdminController({ html });
-
-            config.onBuilt!(app);
-         },
-         "sync"
-      );
-   }
-
-   app.emgr.onEvent(
-      App.Events.AppConfigUpdatedEvent,
-      async ({ params: { app } }) => {
-         saveConfig(app.toJSON(true));
-      },
-      "sync"
-   );
-
-   await app.build();
-
-   if (config.setAdminHtml) {
-      app.registerAdminController({ html });
-   }
-
-   if (!cachedConfig) {
-      saveConfig(app.toJSON(true));
-   }
-
-   return app;
-}
-
-export class DurableBkndApp extends DurableObject {
-   protected id = Math.random().toString(36).slice(2);
-   protected app?: App;
-   protected interval?: any;
-
-   async fire(
-      request: Request,
-      options: {
-         config: CreateAppConfig;
-         html?: string;
-         keepAliveSeconds?: number;
-         setAdminHtml?: boolean;
-      }
-   ) {
-      let buildtime = 0;
-      if (!this.app) {
-         const start = performance.now();
-         const config = options.config;
-
-         // change protocol to websocket if libsql
-         if (
-            config?.connection &&
-            "type" in config.connection &&
-            config.connection.type === "libsql"
-         ) {
-            config.connection.config.protocol = "wss";
-         }
-
-         this.app = App.create(config);
-         this.app.emgr.onEvent(
-            App.Events.AppBuiltEvent,
-            async ({ params: { app } }) => {
-               app.modules.server.get("/__do", async (c) => {
-                  // @ts-ignore
-                  const context: any = c.req.raw.cf ? c.req.raw.cf : c.env.cf;
-                  return c.json({
-                     id: this.id,
-                     keepAlive: options?.keepAliveSeconds,
-                     colo: context.colo
-                  });
-               });
-            },
-            "sync"
-         );
-
-         await this.app.build();
-
-         buildtime = performance.now() - start;
-      }
-
-      if (options?.keepAliveSeconds) {
-         this.keepAlive(options.keepAliveSeconds);
-      }
-
-      console.log("id", this.id);
-      const res = await this.app!.fetch(request);
-      const headers = new Headers(res.headers);
-      headers.set("X-BuildTime", buildtime.toString());
-      headers.set("X-DO-ID", this.id);
-
-      return new Response(res.body, {
-         status: res.status,
-         statusText: res.statusText,
-         headers
-      });
-   }
-
-   protected keepAlive(seconds: number) {
-      console.log("keep alive for", seconds);
-      if (this.interval) {
-         console.log("clearing, there is a new");
-         clearInterval(this.interval);
-      }
-
-      let i = 0;
-      this.interval = setInterval(() => {
-         i += 1;
-         //console.log("keep-alive", i);
-         if (i === seconds) {
-            console.log("cleared");
-            clearInterval(this.interval);
-
-            // ping every 30 seconds
-         } else if (i % 30 === 0) {
-            console.log("ping");
-            this.app?.modules.ctx().connection.ping();
-         }
-      }, 1000);
-   }
-}

app/src/adapter/cloudflare/index.ts

@@ -1 +1,4 @@
 export * from "./cloudflare-workers.adapter";
+export { makeApp, getFresh, getWarm } from "./modes/fresh";
+export { getCached } from "./modes/cached";
+export { DurableBkndApp, getDurable } from "./modes/durable";

app/src/adapter/cloudflare/modes/cached.ts

@@ -0,0 +1,48 @@
+import { createRuntimeApp } from "adapter";
+import { App } from "bknd";
+import type { CloudflareBkndConfig, Context } from "../index";
+
+export async function getCached(config: CloudflareBkndConfig, { env, ctx }: Context) {
+   const { kv } = config.bindings?.(env)!;
+   if (!kv) throw new Error("kv namespace is not defined in cloudflare.bindings");
+   const key = config.key ?? "app";
+
+   const cachedConfig = await kv.get(key);
+   const initialConfig = cachedConfig ? JSON.parse(cachedConfig) : undefined;
+
+   async function saveConfig(__config: any) {
+      ctx.waitUntil(kv!.put(key, JSON.stringify(__config)));
+   }
+
+   const app = await createRuntimeApp(
+      {
+         ...config,
+         initialConfig,
+         onBuilt: async (app) => {
+            app.module.server.client.get("/__bknd/cache", async (c) => {
+               await kv.delete(key);
+               return c.json({ message: "Cache cleared" });
+            });
+            await config.onBuilt?.(app);
+         },
+         beforeBuild: async (app) => {
+            app.emgr.onEvent(
+               App.Events.AppConfigUpdatedEvent,
+               async ({ params: { app } }) => {
+                  saveConfig(app.toJSON(true));
+               },
+               "sync"
+            );
+            await config.beforeBuild?.(app);
+         },
+         adminOptions: { html: config.html }
+      },
+      env
+   );
+
+   if (!cachedConfig) {
+      saveConfig(app.toJSON(true));
+   }
+
+   return app;
+}

app/src/adapter/cloudflare/modes/durable.ts

@@ -0,0 +1,133 @@
+import { DurableObject } from "cloudflare:workers";
+import { createRuntimeApp } from "adapter";
+import type { CloudflareBkndConfig, Context } from "adapter/cloudflare";
+import type { App, CreateAppConfig } from "bknd";
+
+export async function getDurable(config: CloudflareBkndConfig, ctx: Context) {
+   const { dobj } = config.bindings?.(ctx.env)!;
+   if (!dobj) throw new Error("durable object is not defined in cloudflare.bindings");
+   const key = config.key ?? "app";
+
+   if ([config.onBuilt, config.beforeBuild].some((x) => x)) {
+      console.log("onBuilt and beforeBuild are not supported with DurableObject mode");
+   }
+
+   const start = performance.now();
+
+   const id = dobj.idFromName(key);
+   const stub = dobj.get(id) as unknown as DurableBkndApp;
+
+   const create_config = typeof config.app === "function" ? config.app(ctx.env) : config.app;
+
+   const res = await stub.fire(ctx.request, {
+      config: create_config,
+      html: config.html,
+      keepAliveSeconds: config.keepAliveSeconds,
+      setAdminHtml: config.setAdminHtml
+   });
+
+   const headers = new Headers(res.headers);
+   headers.set("X-TTDO", String(performance.now() - start));
+
+   return new Response(res.body, {
+      status: res.status,
+      statusText: res.statusText,
+      headers
+   });
+}
+
+export class DurableBkndApp extends DurableObject {
+   protected id = Math.random().toString(36).slice(2);
+   protected app?: App;
+   protected interval?: any;
+
+   async fire(
+      request: Request,
+      options: {
+         config: CreateAppConfig;
+         html?: string;
+         keepAliveSeconds?: number;
+         setAdminHtml?: boolean;
+      }
+   ) {
+      let buildtime = 0;
+      if (!this.app) {
+         const start = performance.now();
+         const config = options.config;
+
+         // change protocol to websocket if libsql
+         if (
+            config?.connection &&
+            "type" in config.connection &&
+            config.connection.type === "libsql"
+         ) {
+            config.connection.config.protocol = "wss";
+         }
+
+         this.app = await createRuntimeApp({
+            ...config,
+            onBuilt: async (app) => {
+               app.modules.server.get("/__do", async (c) => {
+                  // @ts-ignore
+                  const context: any = c.req.raw.cf ? c.req.raw.cf : c.env.cf;
+                  return c.json({
+                     id: this.id,
+                     keepAliveSeconds: options?.keepAliveSeconds ?? 0,
+                     colo: context.colo
+                  });
+               });
+
+               await this.onBuilt(app);
+            },
+            adminOptions: { html: options.html },
+            beforeBuild: async (app) => {
+               await this.beforeBuild(app);
+            }
+         });
+
+         buildtime = performance.now() - start;
+      }
+
+      if (options?.keepAliveSeconds) {
+         this.keepAlive(options.keepAliveSeconds);
+      }
+
+      console.log("id", this.id);
+      const res = await this.app!.fetch(request);
+      const headers = new Headers(res.headers);
+      headers.set("X-BuildTime", buildtime.toString());
+      headers.set("X-DO-ID", this.id);
+
+      return new Response(res.body, {
+         status: res.status,
+         statusText: res.statusText,
+         headers
+      });
+   }
+
+   async onBuilt(app: App) {}
+   async beforeBuild(app: App) {}
+
+   protected keepAlive(seconds: number) {
+      console.log("keep alive for", seconds);
+      if (this.interval) {
+         console.log("clearing, there is a new");
+         clearInterval(this.interval);
+      }
+
+      let i = 0;
+      this.interval = setInterval(() => {
+         i += 1;
+         //console.log("keep-alive", i);
+         if (i === seconds) {
+            console.log("cleared");
+            clearInterval(this.interval);
+
+            // ping every 30 seconds
+         } else if (i % 30 === 0) {
+            console.log("ping");
+            this.app?.modules.ctx().connection.ping();
+         }
+      }, 1000);
+   }
+}

app/src/adapter/cloudflare/modes/fresh.ts

@@ -0,0 +1,27 @@
+import { createRuntimeApp } from "adapter";
+import type { App } from "bknd";
+import type { CloudflareBkndConfig, Context } from "../index";
+
+export async function makeApp(config: CloudflareBkndConfig, { env }: Context) {
+   return await createRuntimeApp(
+      {
+         ...config,
+         adminOptions: config.html ? { html: config.html } : undefined
+      },
+      env
+   );
+}
+
+export async function getFresh(config: CloudflareBkndConfig, ctx: Context) {
+   const app = await makeApp(config, ctx);
+   return app.fetch(ctx.request);
+}
+
+let warm_app: App;
+export async function getWarm(config: CloudflareBkndConfig, ctx: Context) {
+   if (!warm_app) {
+      warm_app = await makeApp(config, ctx);
+   }
+
+   return warm_app.fetch(ctx.request);
+}

app/src/adapter/index.ts

@@ -1,40 +1,21 @@
 import type { IncomingMessage } from "node:http";
-import type { App, CreateAppConfig } from "bknd";
+import { App, type CreateAppConfig, registries } from "bknd";
+import { config as $config } from "core";
+import type { MiddlewareHandler } from "hono";
+import { StorageLocalAdapter } from "media/storage/adapters/StorageLocalAdapter";
+import type { AdminControllerOptions } from "modules/server/AdminController";
 
-export type CfBkndModeCache<Env = any> = (env: Env) => {
+export type BkndConfig<Env = any> = CreateAppConfig & {
-   cache: KVNamespace;
+   app?: CreateAppConfig | ((env: Env) => CreateAppConfig);
-   key: string;
-};
-
-export type CfBkndModeDurableObject<Env = any> = (env: Env) => {
-   durableObject: DurableObjectNamespace;
-   key: string;
-   keepAliveSeconds?: number;
-};
-
-export type CloudflareBkndConfig<Env = any> = {
-   mode?: CfBkndModeCache | CfBkndModeDurableObject;
-   forceHttps?: boolean;
-};
-
-// @todo: move to App
-export type BkndConfig<Env = any> = {
-   app: CreateAppConfig | ((env: Env) => CreateAppConfig);
-   setAdminHtml?: boolean;
-   server?: {
-      port?: number;
-      platform?: "node" | "bun";
-   };
-   cloudflare?: CloudflareBkndConfig<Env>;
    onBuilt?: (app: App) => Promise<void>;
+   beforeBuild?: (app: App) => Promise<void>;
+   buildConfig?: Parameters<App["build"]>[0];
 };
 
-export type BkndConfigJson = {
+export type FrameworkBkndConfig<Env = any> = BkndConfig<Env>;
-   app: CreateAppConfig;
+
-   setAdminHtml?: boolean;
+export type RuntimeBkndConfig<Env = any> = BkndConfig<Env> & {
-   server?: {
+   distPath?: string;
-      port?: number;
-   };
 };
 
 export function nodeRequestToRequest(req: IncomingMessage): Request {
@@ -60,3 +41,88 @@ export function nodeRequestToRequest(req: IncomingMessage): Request {
       headers
    });
 }
+
+export function registerLocalMediaAdapter() {
+   registries.media.register("local", StorageLocalAdapter);
+}
+
+export function makeConfig<Env = any>(config: BkndConfig<Env>, env?: Env): CreateAppConfig {
+   let additionalConfig: CreateAppConfig = {};
+   if ("app" in config && config.app) {
+      if (typeof config.app === "function") {
+         if (!env) {
+            throw new Error("env is required when config.app is a function");
+         }
+         additionalConfig = config.app(env);
+      } else {
+         additionalConfig = config.app;
+      }
+   }
+
+   return { ...config, ...additionalConfig };
+}
+
+export async function createFrameworkApp<Env = any>(
+   config: FrameworkBkndConfig,
+   env?: Env
+): Promise<App> {
+   const app = App.create(makeConfig(config, env));
+
+   if (config.onBuilt) {
+      app.emgr.onEvent(
+         App.Events.AppBuiltEvent,
+         async () => {
+            await config.onBuilt?.(app);
+         },
+         "sync"
+      );
+   }
+
+   await config.beforeBuild?.(app);
+   await app.build(config.buildConfig);
+
+   return app;
+}
+
+export async function createRuntimeApp<Env = any>(
+   {
+      serveStatic,
+      registerLocalMedia,
+      adminOptions,
+      ...config
+   }: RuntimeBkndConfig & {
+      serveStatic?: MiddlewareHandler | [string, MiddlewareHandler];
+      registerLocalMedia?: boolean;
+      adminOptions?: AdminControllerOptions | false;
+   },
+   env?: Env
+): Promise<App> {
+   if (registerLocalMedia) {
+      registerLocalMediaAdapter();
+   }
+
+   const app = App.create(makeConfig(config, env));
+
+   app.emgr.onEvent(
+      App.Events.AppBuiltEvent,
+      async () => {
+         if (serveStatic) {
+            const [path, handler] = Array.isArray(serveStatic)
+               ? serveStatic
+               : [$config.server.assets_path + "*", serveStatic];
+            app.modules.server.get(path, handler);
+         }
+
+         await config.onBuilt?.(app);
+         if (adminOptions !== false) {
+            app.registerAdminController(adminOptions);
+         }
+      },
+      "sync"
+   );
+
+   await config.beforeBuild?.(app);
+   await app.build(config.buildConfig);
+
+   return app;
+}

app/src/adapter/nextjs/nextjs.adapter.ts

@@ -1,6 +1,10 @@
 import type { IncomingMessage, ServerResponse } from "node:http";
-import { Api, App, type CreateAppConfig } from "bknd";
+import { Api, type App } from "bknd";
-import { nodeRequestToRequest } from "../index";
+import { type FrameworkBkndConfig, createFrameworkApp, nodeRequestToRequest } from "../index";
+
+export type NextjsBkndConfig = FrameworkBkndConfig & {
+   cleanSearch?: string[];
+};
 
 type GetServerSidePropsContext = {
    req: IncomingMessage;
@@ -18,7 +22,6 @@ type GetServerSidePropsContext = {
 
 export function createApi({ req }: GetServerSidePropsContext) {
    const request = nodeRequestToRequest(req);
-   //console.log("createApi:request.headers", request.headers);
    return new Api({
       host: new URL(request.url).origin,
       headers: request.headers
@@ -31,10 +34,13 @@ export function withApi<T>(handler: (ctx: GetServerSidePropsContext & { api: Api
    };
 }
 
-function getCleanRequest(req: Request) {
+function getCleanRequest(
-   // clean search params from "route" attribute
+   req: Request,
+   { cleanSearch = ["route"] }: Pick<NextjsBkndConfig, "cleanSearch">
+) {
    const url = new URL(req.url);
-   url.searchParams.delete("route");
+   cleanSearch?.forEach((k) => url.searchParams.delete(k));
+
    return new Request(url.toString(), {
       method: req.method,
       headers: req.headers,
@@ -43,13 +49,12 @@ function getCleanRequest(req: Request) {
 }
 
 let app: App;
-export function serve(config: CreateAppConfig) {
+export function serve({ cleanSearch, ...config }: NextjsBkndConfig = {}) {
    return async (req: Request) => {
       if (!app) {
-         app = App.create(config);
+         app = await createFrameworkApp(config);
-         await app.build();
       }
-      const request = getCleanRequest(req);
+      const request = getCleanRequest(req, { cleanSearch });
       return app.fetch(request, process.env);
    };
 }

app/src/adapter/node/index.ts

@@ -1,59 +1,6 @@
-import path from "node:path";
+export * from "./node.adapter";
-import { serve as honoServe } from "@hono/node-server";
+export {
-import { serveStatic } from "@hono/node-server/serve-static";
+   StorageLocalAdapter,
-import { App, type CreateAppConfig } from "bknd";
+   type LocalAdapterConfig
-
+} from "../../media/storage/adapters/StorageLocalAdapter";
-export type NodeAdapterOptions = CreateAppConfig & {
+export { registerLocalMediaAdapter } from "../index";
-   relativeDistPath?: string;
-   port?: number;
-   hostname?: string;
-   listener?: Parameters<typeof honoServe>[1];
-};
-
-export function serve({
-   relativeDistPath,
-   port = 1337,
-   hostname,
-   listener,
-   ...config
-}: NodeAdapterOptions = {}) {
-   const root = path.relative(
-      process.cwd(),
-      path.resolve(relativeDistPath ?? "./node_modules/bknd/dist", "static")
-   );
-   let app: App;
-
-   honoServe(
-      {
-         port,
-         hostname,
-         fetch: async (req: Request) => {
-            if (!app) {
-               app = App.create(config);
-
-               app.emgr.on(
-                  "app-built",
-                  async () => {
-                     app.modules.server.get(
-                        "/*",
-                        serveStatic({
-                           root
-                        })
-                     );
-                     app.registerAdminController();
-                  },
-                  "sync"
-               );
-
-               await app.build();
-            }
-
-            return app.fetch(req);
-         }
-      },
-      (connInfo) => {
-         console.log(`Server is running on http://localhost:${connInfo.port}`);
-         listener?.(connInfo);
-      }
-   );
-}

app/src/adapter/node/node.adapter.ts

@@ -0,0 +1,55 @@
+import path from "node:path";
+import { serve as honoServe } from "@hono/node-server";
+import { serveStatic } from "@hono/node-server/serve-static";
+import type { App } from "bknd";
+import { config as $config } from "core";
+import { type RuntimeBkndConfig, createRuntimeApp } from "../index";
+
+export type NodeBkndConfig = RuntimeBkndConfig & {
+   port?: number;
+   hostname?: string;
+   listener?: Parameters<typeof honoServe>[1];
+   /** @deprecated */
+   relativeDistPath?: string;
+};
+
+export function serve({
+   distPath,
+   relativeDistPath,
+   port = $config.server.default_port,
+   hostname,
+   listener,
+   ...config
+}: NodeBkndConfig = {}) {
+   const root = path.relative(
+      process.cwd(),
+      path.resolve(distPath ?? relativeDistPath ?? "./node_modules/bknd/dist", "static")
+   );
+   if (relativeDistPath) {
+      console.warn("relativeDistPath is deprecated, please use distPath instead");
+   }
+
+   let app: App;
+
+   honoServe(
+      {
+         port,
+         hostname,
+         fetch: async (req: Request) => {
+            if (!app) {
+               app = await createRuntimeApp({
+                  ...config,
+                  registerLocalMedia: true,
+                  serveStatic: serveStatic({ root })
+               });
+            }
+
+            return app.fetch(req);
+         }
+      },
+      (connInfo) => {
+         console.log(`Server is running on http://localhost:${connInfo.port}`);
+         listener?.(connInfo);
+      }
+   );
+}

app/src/adapter/remix/remix.adapter.ts

@@ -1,11 +1,13 @@
-import { App, type CreateAppConfig } from "bknd";
+import { type FrameworkBkndConfig, createFrameworkApp } from "adapter";
+import type { App } from "bknd";
+
+export type RemixBkndConfig = FrameworkBkndConfig;
 
 let app: App;
-export function serve(config: CreateAppConfig) {
+export function serve(config: RemixBkndConfig = {}) {
    return async (args: { request: Request }) => {
       if (!app) {
-         app = App.create(config);
+         app = await createFrameworkApp(config);
-         await app.build();
       }
       return app.fetch(args.request);
    };

app/src/adapter/vite/dev-server-config.ts

@@ -0,0 +1,14 @@
+export const devServerConfig = {
+   entry: "./server.ts",
+   exclude: [
+      /.*\.tsx?($|\?)/,
+      /^(?!.*\/__admin).*\.(s?css|less)($|\?)/,
+      // exclude except /api
+      /^(?!.*\/api).*\.(ico|mp4|jpg|jpeg|svg|png|vtt|mp3|js)($|\?)/,
+      /^\/@.+$/,
+      /\/components.*?\.json.*/, // @todo: improve
+      /^\/(public|assets|static)\/.+/,
+      /^\/node_modules\/.*/
+   ] as any,
+   injectClientScript: false
+} as const;

app/src/adapter/vite/vite.adapter.ts

@@ -1,50 +1,81 @@
 import { serveStatic } from "@hono/node-server/serve-static";
-import type { BkndConfig } from "bknd";
+import { type DevServerOptions, default as honoViteDevServer } from "@hono/vite-dev-server";
-import { App } from "bknd";
+import { type RuntimeBkndConfig, createRuntimeApp } from "adapter";
+import type { App } from "bknd";
+import { devServerConfig } from "./dev-server-config";
 
-function createApp(config: BkndConfig, env: any) {
+export type ViteBkndConfig<Env = any> = RuntimeBkndConfig<Env> & {
-   const create_config = typeof config.app === "function" ? config.app(env) : config.app;
+   mode?: "cached" | "fresh";
-   return App.create(create_config);
+   setAdminHtml?: boolean;
-}
+   forceDev?: boolean | { mainPath: string };
+   html?: string;
+};
 
-function setAppBuildListener(app: App, config: BkndConfig, html?: string) {
+export function addViteScript(html: string, addBkndContext: boolean = true) {
-   app.emgr.on(
+   return html.replace(
-      "app-built",
+      "</head>",
-      async () => {
+      `<script type="module">
-         await config.onBuilt?.(app);
+import RefreshRuntime from "/@react-refresh"
-         if (config.setAdminHtml) {
+RefreshRuntime.injectIntoGlobalHook(window)
-            app.registerAdminController({ html, forceDev: true });
+window.$RefreshReg$ = () => {}
-            app.module.server.client.get("/assets/*", serveStatic({ root: "./" }));
+window.$RefreshSig$ = () => (type) => type
-         }
+window.__vite_plugin_react_preamble_installed__ = true
-      },
+</script>
-      "sync"
+<script type="module" src="/@vite/client"></script>
+${addBkndContext ? "<!-- BKND_CONTEXT -->" : ""}
+</head>`
    );
 }
 
-export async function serveFresh(config: BkndConfig, _html?: string) {
+async function createApp(config: ViteBkndConfig = {}, env?: any) {
+   return await createRuntimeApp(
+      {
+         ...config,
+         registerLocalMedia: true,
+         adminOptions:
+            config.setAdminHtml === false
+               ? undefined
+               : {
+                    html: config.html,
+                    forceDev: config.forceDev ?? {
+                       mainPath: "/src/main.tsx"
+                    }
+                 },
+         serveStatic: ["/assets/*", serveStatic({ root: config.distPath ?? "./" })]
+      },
+      env
+   );
+}
+
+export function serveFresh(config: Omit<ViteBkndConfig, "mode"> = {}) {
    return {
       async fetch(request: Request, env: any, ctx: ExecutionContext) {
-         const app = createApp(config, env);
+         const app = await createApp(config, env);
-
-         setAppBuildListener(app, config, _html);
-         await app.build();
-
          return app.fetch(request, env, ctx);
       }
    };
 }
 
 let app: App;
-export async function serveCached(config: BkndConfig, _html?: string) {
+export function serveCached(config: Omit<ViteBkndConfig, "mode"> = {}) {
    return {
       async fetch(request: Request, env: any, ctx: ExecutionContext) {
          if (!app) {
-            app = createApp(config, env);
+            app = await createApp(config, env);
-            setAppBuildListener(app, config, _html);
-            await app.build();
          }
 
          return app.fetch(request, env, ctx);
       }
    };
 }
+
+export function serve({ mode, ...config }: ViteBkndConfig = {}) {
+   return mode === "fresh" ? serveFresh(config) : serveCached(config);
+}
+
+export function devServer(options: DevServerOptions) {
+   return honoViteDevServer({
+      ...devServerConfig,
+      ...options
+   });
+}

app/src/auth/AppAuth.ts

@@ -1,21 +1,30 @@
-import { type AuthAction, Authenticator, type ProfileExchange, Role, type Strategy } from "auth";
+import {
-import { Exception } from "core";
+   type AuthAction,
+   AuthPermissions,
+   Authenticator,
+   type ProfileExchange,
+   Role,
+   type Strategy
+} from "auth";
+import type { PasswordStrategy } from "auth/authenticate/strategies";
+import { type DB, Exception, type PrimaryFieldType } from "core";
 import { type Static, secureRandomString, transformObject } from "core/utils";
-import { type Entity, EntityIndex, type EntityManager } from "data";
+import type { Entity, EntityManager } from "data";
-import { type FieldSchema, entity, enumm, make, text } from "data/prototype";
+import { type FieldSchema, em, entity, enumm, text } from "data/prototype";
 import { pick } from "lodash-es";
 import { Module } from "modules/Module";
 import { AuthController } from "./api/AuthController";
 import { type AppAuthSchema, STRATEGIES, authConfigSchema } from "./auth-schema";
 
 export type UserFieldSchema = FieldSchema<typeof AppAuth.usersFields>;
-declare global {
+declare module "core" {
    interface DB {
-      users: UserFieldSchema;
+      users: { id: PrimaryFieldType } & UserFieldSchema;
    }
 }
 
 type AuthSchema = Static<typeof authConfigSchema>;
+export type CreateUserPayload = { email: string; password: string; [key: string]: any };
 
 export class AppAuth extends Module<typeof authConfigSchema> {
    private _authenticator?: Authenticator;
@@ -35,8 +44,12 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       return to;
    }
 
+   get enabled() {
+      return this.config.enabled;
+   }
+
    override async build() {
-      if (!this.config.enabled) {
+      if (!this.enabled) {
          this.setBuilt();
          return;
       }
@@ -71,8 +84,8 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       super.setBuilt();
 
       this._controller = new AuthController(this);
-      //this.ctx.server.use(controller.getMiddleware);
       this.ctx.server.route(this.config.basepath, this._controller.getController());
+      this.ctx.guard.registerPermissions(Object.values(AuthPermissions));
    }
 
    get controller(): AuthController {
@@ -83,14 +96,6 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       return this._controller;
    }
 
-   getMiddleware() {
-      if (!this.config.enabled) {
-         return;
-      }
-
-      return new AuthController(this).getMiddleware;
-   }
-
    getSchema() {
       return authConfigSchema;
    }
@@ -100,7 +105,7 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       return this._authenticator!;
    }
 
-   get em(): EntityManager<DB> {
+   get em(): EntityManager {
       return this.ctx.em as any;
    }
 
@@ -110,12 +115,12 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       identifier: string,
       profile: ProfileExchange
    ): Promise<any> {
-      console.log("***** AppAuth:resolveUser", {
+      /*console.log("***** AppAuth:resolveUser", {
          action,
          strategy: strategy.getName(),
          identifier,
          profile
-      });
+      });*/
       if (!this.config.allow_register && action === "register") {
          throw new Exception("Registration is not allowed", 403);
       }
@@ -136,12 +141,12 @@ export class AppAuth extends Module<typeof authConfigSchema> {
    }
 
    private filterUserData(user: any) {
-      console.log(
+      /*console.log(
          "--filterUserData",
          user,
          this.config.jwt.fields,
          pick(user, this.config.jwt.fields)
-      );
+      );*/
       return pick(user, this.config.jwt.fields);
    }
 
@@ -160,23 +165,25 @@ export class AppAuth extends Module<typeof authConfigSchema> {
 
       const users = this.getUsersEntity();
       this.toggleStrategyValueVisibility(true);
-      const result = await this.em.repo(users).findOne({ email: profile.email! });
+      const result = await this.em
+         .repo(users as unknown as "users")
+         .findOne({ email: profile.email! });
       this.toggleStrategyValueVisibility(false);
       if (!result.data) {
          throw new Exception("User not found", 404);
       }
-      console.log("---login data", result.data, result);
+      //console.log("---login data", result.data, result);
 
       // compare strategy and identifier
-      console.log("strategy comparison", result.data.strategy, strategy.getName());
+      //console.log("strategy comparison", result.data.strategy, strategy.getName());
       if (result.data.strategy !== strategy.getName()) {
-         console.log("!!! User registered with different strategy");
+         //console.log("!!! User registered with different strategy");
          throw new Exception("User registered with different strategy");
       }
 
-      console.log("identifier comparison", result.data.strategy_value, identifier);
+      //console.log("identifier comparison", result.data.strategy_value, identifier);
       if (result.data.strategy_value !== identifier) {
-         console.log("!!! Invalid credentials");
+         //console.log("!!! Invalid credentials");
          throw new Exception("Invalid credentials");
       }
 
@@ -197,7 +204,7 @@ export class AppAuth extends Module<typeof authConfigSchema> {
          throw new Exception("User already exists");
       }
 
-      const payload = {
+      const payload: any = {
          ...profile,
          strategy: strategy.getName(),
          strategy_value: identifier
@@ -217,10 +224,23 @@ export class AppAuth extends Module<typeof authConfigSchema> {
    }
 
    private toggleStrategyValueVisibility(visible: boolean) {
-      const field = this.getUsersEntity().field("strategy_value")!;
+      const toggle = (name: string, visible: boolean) => {
+         const field = this.getUsersEntity().field(name)!;
+
+         if (visible) {
+            field.config.hidden = false;
+            field.config.fillable = true;
+         } else {
+            // reset to normal
+            const template = AppAuth.usersFields.strategy_value.config;
+            field.config.hidden = template.hidden;
+            field.config.fillable = template.fillable;
+         }
+      };
+
+      toggle("strategy_value", visible);
+      toggle("strategy", visible);
 
-      field.config.hidden = !visible;
-      field.config.fillable = visible;
       // @todo: think about a PasswordField that automatically hashes on save?
    }
 
@@ -235,7 +255,10 @@ export class AppAuth extends Module<typeof authConfigSchema> {
 
    static usersFields = {
       email: text().required(),
-      strategy: text({ fillable: ["create"], hidden: ["form"] }).required(),
+      strategy: text({
+         fillable: ["create"],
+         hidden: ["update", "form"]
+      }).required(),
       strategy_value: text({
          fillable: ["create"],
          hidden: ["read", "table", "update", "form"]
@@ -244,46 +267,49 @@ export class AppAuth extends Module<typeof authConfigSchema> {
    };
 
    registerEntities() {
-      const users = this.getUsersEntity();
+      const users = this.getUsersEntity(true);
-
+      this.ensureSchema(
-      if (!this.em.hasEntity(users.name)) {
+         em(
-         this.em.addEntity(users);
+            {
-      } else {
+               [users.name as "users"]: users
-         // if exists, check all fields required are there
+            },
-         // @todo: add to context: "needs sync" flag
+            ({ index }, { users }) => {
-         const _entity = this.getUsersEntity(true);
+               index(users).on(["email"], true).on(["strategy"]).on(["strategy_value"]);
-         for (const field of _entity.fields) {
-            const _field = users.field(field.name);
-            if (!_field) {
-               users.addField(field);
             }
-         }
+         )
-      }
+      );
-
-      const indices = [
-         new EntityIndex(users, [users.field("email")!], true),
-         new EntityIndex(users, [users.field("strategy")!]),
-         new EntityIndex(users, [users.field("strategy_value")!])
-      ];
-      indices.forEach((index) => {
-         if (!this.em.hasIndex(index)) {
-            this.em.addIndex(index);
-         }
-      });
 
       try {
          const roles = Object.keys(this.config.roles ?? {});
-         const field = make("role", enumm({ enum: roles }));
+         this.replaceEntityField(users, "role", enumm({ enum: roles }));
-         this.em.entity(users.name).__experimental_replaceField("role", field);
       } catch (e) {}
 
       try {
          const strategies = Object.keys(this.config.strategies ?? {});
-         const field = make("strategy", enumm({ enum: strategies }));
+         this.replaceEntityField(users, "strategy", enumm({ enum: strategies }));
-         this.em.entity(users.name).__experimental_replaceField("strategy", field);
       } catch (e) {}
    }
 
+   async createUser({ email, password, ...additional }: CreateUserPayload): Promise<DB["users"]> {
+      if (!this.enabled) {
+         throw new Error("Cannot create user, auth not enabled");
+      }
+
+      const strategy = "password";
+      const pw = this.authenticator.strategy(strategy) as PasswordStrategy;
+      const strategy_value = await pw.hash(password);
+      const mutator = this.em.mutator(this.config.entity_name as "users");
+      mutator.__unstable_toggleSystemEntityCreation(false);
+      const { data: created } = await mutator.insertOne({
+         ...(additional as any),
+         email,
+         strategy,
+         strategy_value
+      });
+      mutator.__unstable_toggleSystemEntityCreation(true);
+      return created;
+   }
+
    override toJSON(secrets?: boolean): AppAuthSchema {
       if (!this.config.enabled) {
          return this.configDefault;

app/src/auth/api/AuthApi.ts

@@ -1,4 +1,5 @@
-import type { AppAuthSchema, AppAuthStrategies } from "auth/auth-schema";
+import type { AuthActionResponse } from "auth/api/AuthController";
+import type { AppAuthSchema } from "auth/auth-schema";
 import type { AuthResponse, SafeUser, Strategy } from "auth/authenticate/Authenticator";
 import { type BaseModuleApiOptions, ModuleApi } from "modules/ModuleApi";
 
@@ -13,27 +14,51 @@ export class AuthApi extends ModuleApi<AuthApiOptions> {
       };
    }
 
+   async login(strategy: string, input: any) {
+      const res = await this.post<AuthResponse>([strategy, "login"], input);
+      if (res.ok && res.body.token) {
+         await this.options.onTokenUpdate?.(res.body.token);
+      }
+      return res;
+   }
+
+   async register(strategy: string, input: any) {
+      const res = await this.post<AuthResponse>([strategy, "register"], input);
+      if (res.ok && res.body.token) {
+         await this.options.onTokenUpdate?.(res.body.token);
+      }
+      return res;
+   }
+
+   async actionSchema(strategy: string, action: string) {
+      return this.get<Strategy>([strategy, "actions", action, "schema.json"]);
+   }
+
+   async action(strategy: string, action: string, input: any) {
+      return this.post<AuthActionResponse>([strategy, "actions", action], input);
+   }
+
+   /**
+    * @deprecated use login("password", ...) instead
+    * @param input
+    */
    async loginWithPassword(input: any) {
-      const res = await this.post<AuthResponse>(["password", "login"], input);
+      return this.login("password", input);
-      if (res.res.ok && res.body.token) {
-         await this.options.onTokenUpdate?.(res.body.token);
-      }
-      return res;
    }
 
+   /**
+    * @deprecated use register("password", ...) instead
+    * @param input
+    */
    async registerWithPassword(input: any) {
-      const res = await this.post<AuthResponse>(["password", "register"], input);
+      return this.register("password", input);
-      if (res.res.ok && res.body.token) {
-         await this.options.onTokenUpdate?.(res.body.token);
-      }
-      return res;
    }
 
-   async me() {
+   me() {
       return this.get<{ user: SafeUser | null }>(["me"]);
    }
 
-   async strategies() {
+   strategies() {
       return this.get<Pick<AppAuthSchema, "strategies" | "basepath">>(["strategies"]);
    }
 

app/src/auth/api/AuthController.ts

@@ -1,58 +1,108 @@
-import type { AppAuth } from "auth";
+import { type AppAuth, AuthPermissions, type SafeUser, type Strategy } from "auth";
-import { type ClassController, isDebug } from "core";
+import { TypeInvalidError, parse } from "core/utils";
-import { Hono, type MiddlewareHandler } from "hono";
+import { DataPermissions } from "data";
+import type { Hono } from "hono";
+import { Controller } from "modules/Controller";
+import type { ServerEnv } from "modules/Module";
 
-export class AuthController implements ClassController {
+export type AuthActionResponse = {
-   constructor(private auth: AppAuth) {}
+   success: boolean;
+   action: string;
+   data?: SafeUser;
+   errors?: any;
+};
+
+export class AuthController extends Controller {
+   constructor(private auth: AppAuth) {
+      super();
+   }
 
    get guard() {
       return this.auth.ctx.guard;
    }
 
-   getMiddleware: MiddlewareHandler = async (c, next) => {
+   private registerStrategyActions(strategy: Strategy, mainHono: Hono<ServerEnv>) {
-      // @todo: ONLY HOTFIX
+      const actions = strategy.getActions?.();
-      // middlewares are added for all routes are registered. But we need to make sure that
+      if (!actions) {
-      // only HTML/JSON routes are adding a cookie to the response. Config updates might
+         return;
-      // also use an extension "syntax", e.g. /api/system/patch/data/entities.posts
-      // This middleware should be extracted and added by each Controller individually,
-      // but it requires access to the auth secret.
-      // Note: This doesn't mean endpoints aren't protected, just the cookie is not set.
-      const url = new URL(c.req.url);
-      const last = url.pathname.split("/")?.pop();
-      const ext = last?.includes(".") ? last.split(".")?.pop() : undefined;
-      if (
-         !this.auth.authenticator.isJsonRequest(c) &&
-         ["GET", "HEAD", "OPTIONS"].includes(c.req.method) &&
-         ext &&
-         ["js", "css", "png", "jpg", "jpeg", "svg", "ico"].includes(ext)
-      ) {
-         isDebug() && console.log("Skipping auth", { ext }, url.pathname);
-      } else {
-         const user = await this.auth.authenticator.resolveAuthFromRequest(c);
-         this.auth.ctx.guard.setUserContext(user);
       }
 
-      await next();
+      const { auth, permission } = this.middlewares;
-   };
+      const hono = this.create().use(auth());
 
-   getController(): Hono<any> {
+      const name = strategy.getName();
-      const hono = new Hono();
+      const { create, change } = actions;
+      const em = this.auth.em;
+
+      if (create) {
+         hono.post(
+            "/create",
+            permission([AuthPermissions.createUser, DataPermissions.entityCreate]),
+            async (c) => {
+               try {
+                  const body = await this.auth.authenticator.getBody(c);
+                  const valid = parse(create.schema, body, {
+                     skipMark: true
+                  });
+                  const processed = (await create.preprocess?.(valid)) ?? valid;
+
+                  // @todo: check processed for "role" and check permissions
+                  const mutator = em.mutator(this.auth.config.entity_name as "users");
+                  mutator.__unstable_toggleSystemEntityCreation(false);
+                  const { data: created } = await mutator.insertOne({
+                     ...processed,
+                     strategy: name
+                  });
+                  mutator.__unstable_toggleSystemEntityCreation(true);
+
+                  return c.json({
+                     success: true,
+                     action: "create",
+                     strategy: name,
+                     data: created as unknown as SafeUser
+                  } as AuthActionResponse);
+               } catch (e) {
+                  if (e instanceof TypeInvalidError) {
+                     return c.json(
+                        {
+                           success: false,
+                           errors: e.errors
+                        },
+                        400
+                     );
+                  }
+                  throw e;
+               }
+            }
+         );
+         hono.get("create/schema.json", async (c) => {
+            return c.json(create.schema);
+         });
+      }
+
+      mainHono.route(`/${name}/actions`, hono);
+   }
+
+   override getController() {
+      const { auth } = this.middlewares;
+      const hono = this.create();
       const strategies = this.auth.authenticator.getStrategies();
 
       for (const [name, strategy] of Object.entries(strategies)) {
          //console.log("registering", name, "at", `/${name}`);
          hono.route(`/${name}`, strategy.getController(this.auth.authenticator));
+         this.registerStrategyActions(strategy, hono);
       }
 
-      hono.get("/me", async (c) => {
+      hono.get("/me", auth(), async (c) => {
          if (this.auth.authenticator.isUserLoggedIn()) {
-            return c.json({ user: await this.auth.authenticator.getUser() });
+            return c.json({ user: this.auth.authenticator.getUser() });
          }
 
          return c.json({ user: null }, 403);
       });
 
-      hono.get("/logout", async (c) => {
+      hono.get("/logout", auth(), async (c) => {
          await this.auth.authenticator.logout(c);
          if (this.auth.authenticator.isJsonRequest(c)) {
             return c.json({ ok: true });

app/src/auth/auth-permissions.ts

@@ -0,0 +1,4 @@
+import { Permission } from "core";
+
+export const createUser = new Permission("auth.user.create");
+//export const updateUser = new Permission("auth.user.update");

app/src/auth/auth-schema.ts

@@ -33,6 +33,7 @@ const strategiesSchemaObject = objectTransform(STRATEGIES, (strategy, name) => {
 const strategiesSchema = Type.Union(Object.values(strategiesSchemaObject));
 export type AppAuthStrategies = Static<typeof strategiesSchema>;
 export type AppAuthOAuthStrategy = Static<typeof STRATEGIES.oauth.schema>;
+export type AppAuthCustomOAuthStrategy = Static<typeof STRATEGIES.custom_oauth.schema>;
 
 const guardConfigSchema = Type.Object({
    enabled: Type.Optional(Type.Boolean({ default: false }))

app/src/auth/authenticate/Authenticator.ts

@@ -1,23 +1,31 @@
-import { Exception } from "core";
+import { type DB, Exception } from "core";
 import { addFlashMessage } from "core/server/flash";
 import {
    type Static,
    StringEnum,
-   type TSchema,
+   type TObject,
    Type,
    parse,
-   randomString,
+   runtimeSupports,
    transformObject
 } from "core/utils";
 import type { Context, Hono } from "hono";
 import { deleteCookie, getSignedCookie, setSignedCookie } from "hono/cookie";
 import { sign, verify } from "hono/jwt";
 import type { CookieOptions } from "hono/utils/cookie";
-import { omit } from "lodash-es";
+import type { ServerEnv } from "modules/Module";
 
 type Input = any; // workaround
 export type JWTPayload = Parameters<typeof sign>[0];
 
+export const strategyActions = ["create", "change"] as const;
+export type StrategyActionName = (typeof strategyActions)[number];
+export type StrategyAction<S extends TObject = TObject> = {
+   schema: S;
+   preprocess: (input: unknown) => Promise<Omit<DB["users"], "id" | "strategy">>;
+};
+export type StrategyActions = Partial<Record<StrategyActionName, StrategyAction>>;
+
 // @todo: add schema to interface to ensure proper inference
 export interface Strategy {
    getController: (auth: Authenticator) => Hono<any>;
@@ -25,6 +33,7 @@ export interface Strategy {
    getMode: () => "form" | "external";
    getName: () => string;
    toJSON: (secrets?: boolean) => any;
+   getActions?: () => StrategyActions;
 }
 
 export type User = {
@@ -67,6 +76,9 @@ export const cookieConfig = Type.Partial(
    { default: {}, additionalProperties: false }
 );
 
+// @todo: maybe add a config to not allow cookie/api tokens to be used interchangably?
+// see auth.integration test for further details
+
 export const jwtConfig = Type.Object(
    {
       // @todo: autogenerate a secret if not present. But it must be persisted from AppAuth
@@ -98,7 +110,13 @@ export type AuthUserResolver = (
 export class Authenticator<Strategies extends Record<string, Strategy> = Record<string, Strategy>> {
    private readonly strategies: Strategies;
    private readonly config: AuthConfig;
-   private _user: SafeUser | undefined;
+   private _claims:
+      | undefined
+      | (SafeUser & {
+           iat: number;
+           iss?: string;
+           exp?: number;
+        });
    private readonly userResolver: AuthUserResolver;
 
    constructor(strategies: Strategies, userResolver?: AuthUserResolver, config?: AuthConfig) {
@@ -131,16 +149,18 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
    }
 
    isUserLoggedIn(): boolean {
-      return this._user !== undefined;
+      return this._claims !== undefined;
    }
 
-   getUser() {
+   getUser(): SafeUser | undefined {
-      return this._user;
+      if (!this._claims) return;
+
+      const { iat, exp, iss, ...user } = this._claims;
+      return user;
    }
 
-   // @todo: determine what to do exactly
+   resetUser() {
-   __setUserNull() {
+      this._claims = undefined;
-      this._user = undefined;
    }
 
    strategy<
@@ -154,6 +174,7 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       }
    }
 
+   // @todo: add jwt tests
    async jwt(user: Omit<User, "password">): Promise<string> {
       const prohibited = ["password"];
       for (const prop of prohibited) {
@@ -200,11 +221,11 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
             }
          }
 
-         this._user = omit(payload, ["iat", "exp", "iss"]) as SafeUser;
+         this._claims = payload as any;
          return true;
       } catch (e) {
-         this._user = undefined;
+         this.resetUser();
-         console.error(e);
+         //console.error(e);
       }
 
       return false;
@@ -220,38 +241,48 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
    }
 
    private async getAuthCookie(c: Context): Promise<string | undefined> {
-      const secret = this.config.jwt.secret;
+      try {
+         const secret = this.config.jwt.secret;
+         const token = await getSignedCookie(c, secret, "auth");
+         if (typeof token !== "string") {
+            return undefined;
+         }
+
+         return token;
+      } catch (e: any) {
+         if (e instanceof Error) {
+            console.error("[Error:getAuthCookie]", e.message);
+         }
 
-      const token = await getSignedCookie(c, secret, "auth");
-      if (typeof token !== "string") {
-         await deleteCookie(c, "auth", this.cookieOptions);
          return undefined;
       }
-
-      return token;
    }
 
    async requestCookieRefresh(c: Context) {
       if (this.config.cookie.renew) {
          const token = await this.getAuthCookie(c);
          if (token) {
-            console.log("renewing cookie", c.req.url);
             await this.setAuthCookie(c, token);
          }
       }
    }
 
-   private async setAuthCookie(c: Context, token: string) {
+   private async setAuthCookie(c: Context<ServerEnv>, token: string) {
       const secret = this.config.jwt.secret;
       await setSignedCookie(c, "auth", token, secret, this.cookieOptions);
    }
 
+   private async deleteAuthCookie(c: Context) {
+      await deleteCookie(c, "auth", this.cookieOptions);
+   }
+
    async logout(c: Context) {
       const cookie = await this.getAuthCookie(c);
       if (cookie) {
-         await deleteCookie(c, "auth", this.cookieOptions);
+         await this.deleteAuthCookie(c);
          await addFlashMessage(c, "Signed out", "info");
       }
+      this.resetUser();
    }
 
    // @todo: move this to a server helper
@@ -260,18 +291,39 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       return c.req.header("Content-Type") === "application/json";
    }
 
+   async getBody(c: Context) {
+      if (this.isJsonRequest(c)) {
+         return await c.req.json();
+      } else {
+         return Object.fromEntries((await c.req.formData()).entries());
+      }
+   }
+
+   private getSuccessPath(c: Context) {
+      const p = (this.config.cookie.pathSuccess ?? "/").replace(/\/+$/, "/");
+
+      // nextjs doesn't support non-fq urls
+      // but env could be proxied (stackblitz), so we shouldn't fq every url
+      if (!runtimeSupports("redirects_non_fq")) {
+         return new URL(c.req.url).origin + p;
+      }
+
+      return p;
+   }
+
    async respond(c: Context, data: AuthResponse | Error | any, redirect?: string) {
       if (this.isJsonRequest(c)) {
          return c.json(data);
       }
 
-      const successPath = this.config.cookie.pathSuccess ?? "/";
+      const successUrl = this.getSuccessPath(c);
-      const successUrl = new URL(c.req.url).origin + successPath.replace(/\/+$/, "/");
+      const referer = redirect ?? c.req.header("Referer") ?? successUrl;
-      const referer = new URL(redirect ?? c.req.header("Referer") ?? successUrl);
+      //console.log("auth respond", { redirect, successUrl, successPath });
 
       if ("token" in data) {
          await this.setAuthCookie(c, data.token);
          // can't navigate to "/" – doesn't work on nextjs
+         //console.log("auth success, redirecting to", successUrl);
          return c.redirect(successUrl);
       }
 
@@ -281,6 +333,7 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       }
 
       await addFlashMessage(c, message, "error");
+      //console.log("auth failed, redirecting to", referer);
       return c.redirect(referer);
    }
 
@@ -296,7 +349,7 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
 
       if (token) {
          await this.verify(token);
-         return this._user;
+         return this.getUser();
       }
 
       return undefined;
@@ -310,3 +363,13 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       };
    }
 }
+
+export function createStrategyAction<S extends TObject>(
+   schema: S,
+   preprocess: (input: Static<S>) => Promise<Partial<DB["users"]>>
+) {
+   return {
+      schema,
+      preprocess
+   } as StrategyAction<S>;
+}

app/src/auth/authenticate/strategies/PasswordStrategy.ts

@@ -2,6 +2,7 @@ import type { Authenticator, Strategy } from "auth";
 import { type Static, StringEnum, Type, parse } from "core/utils";
 import { hash } from "core/utils";
 import { type Context, Hono } from "hono";
+import { type StrategyAction, type StrategyActions, createStrategyAction } from "../Authenticator";
 
 type LoginSchema = { username: string; password: string } | { email: string; password: string };
 type RegisterSchema = { email: string; password: string; [key: string]: any };
@@ -54,17 +55,9 @@ export class PasswordStrategy implements Strategy {
    getController(authenticator: Authenticator): Hono<any> {
       const hono = new Hono();
 
-      async function getBody(c: Context) {
-         if (authenticator.isJsonRequest(c)) {
-            return await c.req.json();
-         } else {
-            return Object.fromEntries((await c.req.formData()).entries());
-         }
-      }
-
       return hono
          .post("/login", async (c) => {
-            const body = await getBody(c);
+            const body = await authenticator.getBody(c);
 
             try {
                const payload = await this.login(body);
@@ -76,7 +69,7 @@ export class PasswordStrategy implements Strategy {
             }
          })
          .post("/register", async (c) => {
-            const body = await getBody(c);
+            const body = await authenticator.getBody(c);
 
             const payload = await this.register(body);
             const data = await authenticator.resolve("register", this, payload.password, payload);
@@ -85,6 +78,27 @@ export class PasswordStrategy implements Strategy {
          });
    }
 
+   getActions(): StrategyActions {
+      return {
+         create: createStrategyAction(
+            Type.Object({
+               email: Type.String({
+                  pattern: "^[\\w-\\.]+@([\\w-]+\\.)+[\\w-]{2,4}$"
+               }),
+               password: Type.String({
+                  minLength: 8 // @todo: this should be configurable
+               })
+            }),
+            async ({ password, ...input }) => {
+               return {
+                  ...input,
+                  strategy_value: await this.hash(password)
+               };
+            }
+         )
+      };
+   }
+
    getSchema() {
       return schema;
    }

app/src/auth/authorize/Guard.ts

@@ -98,12 +98,16 @@ export class Guard {
       if (this.user && typeof this.user.role === "string") {
          const role = this.roles?.find((role) => role.name === this.user?.role);
          if (role) {
-            debug && console.log("guard: role found", this.user.role);
+            debug && console.log("guard: role found", [this.user.role]);
             return role;
          }
       }
 
-      debug && console.log("guard: role not found", this.user, this.user?.role);
+      debug &&
+         console.log("guard: role not found", {
+            user: this.user,
+            role: this.user?.role
+         });
       return this.getDefaultRole();
    }
 

app/src/auth/index.ts

@@ -19,3 +19,5 @@ export { AppAuth, type UserFieldSchema } from "./AppAuth";
 
 export { Guard, type GuardUserContext, type GuardConfig } from "./authorize/Guard";
 export { Role } from "./authorize/Role";
+
+export * as AuthPermissions from "./auth-permissions";

app/src/auth/middlewares.ts

@@ -0,0 +1,108 @@
+import type { Permission } from "core";
+import { patternMatch } from "core/utils";
+import type { Context } from "hono";
+import { createMiddleware } from "hono/factory";
+import type { ServerEnv } from "modules/Module";
+
+function getPath(reqOrCtx: Request | Context) {
+   const req = reqOrCtx instanceof Request ? reqOrCtx : reqOrCtx.req.raw;
+   return new URL(req.url).pathname;
+}
+
+export function shouldSkip(c: Context<ServerEnv>, skip?: (string | RegExp)[]) {
+   if (c.get("auth_skip")) return true;
+
+   const req = c.req.raw;
+   if (!skip) return false;
+
+   const path = getPath(req);
+   const result = skip.some((s) => patternMatch(path, s));
+
+   c.set("auth_skip", result);
+   return result;
+}
+
+export const auth = (options?: {
+   skip?: (string | RegExp)[];
+}) =>
+   createMiddleware<ServerEnv>(async (c, next) => {
+      const app = c.get("app");
+      const guard = app?.modules.ctx().guard;
+      const authenticator = app?.module.auth.authenticator;
+
+      let skipped = shouldSkip(c, options?.skip) || !app?.module.auth.enabled;
+
+      // make sure to only register once
+      if (c.get("auth_registered")) {
+         skipped = true;
+         console.warn(`auth middleware already registered for ${getPath(c)}`);
+      } else {
+         c.set("auth_registered", true);
+
+         if (!skipped) {
+            const resolved = c.get("auth_resolved");
+            if (!resolved) {
+               if (!app?.module.auth.enabled) {
+                  guard?.setUserContext(undefined);
+               } else {
+                  guard?.setUserContext(await authenticator?.resolveAuthFromRequest(c));
+                  c.set("auth_resolved", true);
+               }
+            }
+         }
+      }
+
+      await next();
+
+      if (!skipped) {
+         // renew cookie if applicable
+         authenticator?.requestCookieRefresh(c);
+      }
+
+      // release
+      guard?.setUserContext(undefined);
+      authenticator?.resetUser();
+      c.set("auth_resolved", false);
+   });
+
+export const permission = (
+   permission: Permission | Permission[],
+   options?: {
+      onGranted?: (c: Context<ServerEnv>) => Promise<Response | void | undefined>;
+      onDenied?: (c: Context<ServerEnv>) => Promise<Response | void | undefined>;
+   }
+) =>
+   // @ts-ignore
+   createMiddleware<ServerEnv>(async (c, next) => {
+      const app = c.get("app");
+      //console.log("skip?", c.get("auth_skip"));
+
+      // in tests, app is not defined
+      if (!c.get("auth_registered") || !app) {
+         const msg = `auth middleware not registered, cannot check permissions for ${getPath(c)}`;
+         if (app?.module.auth.enabled) {
+            throw new Error(msg);
+         } else {
+            console.warn(msg);
+         }
+      } else if (!c.get("auth_skip")) {
+         const guard = app.modules.ctx().guard;
+         const permissions = Array.isArray(permission) ? permission : [permission];
+
+         if (options?.onGranted || options?.onDenied) {
+            let returned: undefined | void | Response;
+            if (permissions.every((p) => guard.granted(p))) {
+               returned = await options?.onGranted?.(c);
+            } else {
+               returned = await options?.onDenied?.(c);
+            }
+            if (returned instanceof Response) {
+               return returned;
+            }
+         } else {
+            permissions.some((p) => guard.throwUnlessGranted(p));
+         }
+      }
+
+      await next();
+   });

app/src/cli/commands/run/platform.ts

@@ -1,5 +1,6 @@
 import path from "node:path";
 import type { Config } from "@libsql/client/node";
+import { config } from "core";
 import type { MiddlewareHandler } from "hono";
 import open from "open";
 import { fileExists, getRelativeDistPath } from "../../utils/sys";
@@ -26,7 +27,7 @@ export async function serveStatic(server: Platform): Promise<MiddlewareHandler>
 }
 
 export async function attachServeStatic(app: any, platform: Platform) {
-   app.module.server.client.get("/*", await serveStatic(platform));
+   app.module.server.client.get(config.server.assets_path + "*", await serveStatic(platform));
 }
 
 export async function startServer(server: Platform, app: any, options: { port: number }) {

app/src/cli/commands/run/run.ts

@@ -1,8 +1,10 @@
 import type { Config } from "@libsql/client/node";
 import { App, type CreateAppConfig } from "App";
-import type { BkndConfig } from "adapter";
+import { StorageLocalAdapter } from "adapter/node";
-import type { CliCommand } from "cli/types";
+import type { CliBkndConfig, CliCommand } from "cli/types";
 import { Option } from "commander";
+import { config } from "core";
+import { registries } from "modules/registries";
 import {
    PLATFORMS,
    type Platform,
@@ -19,7 +21,7 @@ export const run: CliCommand = (program) => {
       .addOption(
          new Option("-p, --port <port>", "port to run on")
             .env("PORT")
-            .default(1337)
+            .default(config.server.default_port)
             .argParser((v) => Number.parseInt(v))
       )
       .addOption(new Option("-c, --config <config>", "config file"))
@@ -37,6 +39,12 @@ export const run: CliCommand = (program) => {
       .action(action);
 };
 
+// automatically register local adapter
+const local = StorageLocalAdapter.prototype.getName();
+if (!registries.media.has(local)) {
+   registries.media.register(local, StorageLocalAdapter);
+}
+
 type MakeAppConfig = {
    connection?: CreateAppConfig["connection"];
    server?: { platform?: Platform };
@@ -47,8 +55,8 @@ type MakeAppConfig = {
 async function makeApp(config: MakeAppConfig) {
    const app = App.create({ connection: config.connection });
 
-   app.emgr.on(
+   app.emgr.onEvent(
-      "app-built",
+      App.Events.AppBuiltEvent,
       async () => {
          await attachServeStatic(app, config.server?.platform ?? "node");
          app.registerAdminController();
@@ -64,24 +72,23 @@ async function makeApp(config: MakeAppConfig) {
    return app;
 }
 
-export async function makeConfigApp(config: BkndConfig, platform?: Platform) {
+export async function makeConfigApp(config: CliBkndConfig, platform?: Platform) {
    const appConfig = typeof config.app === "function" ? config.app(process.env) : config.app;
    const app = App.create(appConfig);
 
-   app.emgr.on(
+   app.emgr.onEvent(
-      "app-built",
+      App.Events.AppBuiltEvent,
       async () => {
          await attachServeStatic(app, platform ?? "node");
          app.registerAdminController();
 
-         if (config.onBuilt) {
+         await config.onBuilt?.(app);
-            await config.onBuilt(app);
-         }
       },
       "sync"
    );
 
-   await app.build();
+   await config.beforeBuild?.(app);
+   await app.build(config.buildConfig);
    return app;
 }
 
@@ -102,7 +109,7 @@ async function action(options: {
       app = await makeApp({ connection, server: { platform: options.server } });
    } else {
       console.log("Using config from:", configFilePath);
-      const config = (await import(configFilePath).then((m) => m.default)) as BkndConfig;
+      const config = (await import(configFilePath).then((m) => m.default)) as CliBkndConfig;
       app = await makeConfigApp(config, options.server);
    }
 

app/src/cli/commands/user.ts

@@ -1,9 +1,9 @@
 import { password as $password, text as $text } from "@clack/prompts";
+import type { App } from "App";
 import type { PasswordStrategy } from "auth/authenticate/strategies";
-import type { App, BkndConfig } from "bknd";
 import { makeConfigApp } from "cli/commands/run";
 import { getConfigPath } from "cli/commands/run/platform";
-import type { CliCommand } from "cli/types";
+import type { CliBkndConfig, CliCommand } from "cli/types";
 import { Argument } from "commander";
 
 export const user: CliCommand = (program) => {
@@ -21,7 +21,7 @@ async function action(action: "create" | "update", options: any) {
       return;
    }
 
-   const config = (await import(configFilePath).then((m) => m.default)) as BkndConfig;
+   const config = (await import(configFilePath).then((m) => m.default)) as CliBkndConfig;
    const app = await makeConfigApp(config, options.server);
 
    switch (action) {
@@ -35,9 +35,11 @@ async function action(action: "create" | "update", options: any) {
 }
 
 async function create(app: App, options: any) {
-   const config = app.module.auth.toJSON(true);
    const strategy = app.module.auth.authenticator.strategy("password") as PasswordStrategy;
-   const users_entity = config.entity_name;
+
+   if (!strategy) {
+      throw new Error("Password strategy not configured");
+   }
 
    const email = await $text({
       message: "Enter email",
@@ -65,16 +67,11 @@ async function create(app: App, options: any) {
    }
 
    try {
-      const mutator = app.modules.ctx().em.mutator(users_entity);
+      const created = await app.createUser({
-      mutator.__unstable_toggleSystemEntityCreation(false);
-      const res = await mutator.insertOne({
          email,
-         strategy: "password",
+         password: await strategy.hash(password as string)
-         strategy_value: await strategy.hash(password as string)
+      })
-      });
+      console.log("Created:", created);
-      mutator.__unstable_toggleSystemEntityCreation(true);
-
-      console.log("Created:", res.data);
    } catch (e) {
       console.error("Error", e);
    }
@@ -83,7 +80,7 @@ async function create(app: App, options: any) {
 async function update(app: App, options: any) {
    const config = app.module.auth.toJSON(true);
    const strategy = app.module.auth.authenticator.strategy("password") as PasswordStrategy;
-   const users_entity = config.entity_name;
+   const users_entity = config.entity_name as "users";
    const em = app.modules.ctx().em;
 
    const email = (await $text({
@@ -141,4 +138,4 @@ async function update(app: App, options: any) {
    } catch (e) {
       console.error("Error", e);
    }
-}
+}

app/src/cli/types.d.ts

@@ -1,3 +1,14 @@
+import type { CreateAppConfig } from "App";
+import type { FrameworkBkndConfig } from "adapter";
 import type { Command } from "commander";
 
 export type CliCommand = (program: Command) => void;
+
+export type CliBkndConfig<Env = any> = FrameworkBkndConfig & {
+   app: CreateAppConfig | ((env: Env) => CreateAppConfig);
+   setAdminHtml?: boolean;
+   server?: {
+      port?: number;
+      platform?: "node" | "bun";
+   };
+};

app/src/core/config.ts

@@ -5,7 +5,20 @@ import type { Generated } from "kysely";
 
 export type PrimaryFieldType = number | Generated<number>;
 
+export interface DB {
+   // make sure to make unknown as "any"
+   [key: string]: {
+      id: PrimaryFieldType;
+      [key: string]: any;
+   };
+}
+
 export const config = {
+   server: {
+      default_port: 1337,
+      // resetted to root for now, bc bundling with vite
+      assets_path: "/"
+   },
    data: {
       default_primary_field: "id"
    }

app/src/core/errors.ts

@@ -1,6 +1,7 @@
 export class Exception extends Error {
    code = 400;
    override name = "Exception";
+   protected _context = undefined;
 
    constructor(message: string, code?: number) {
       super(message);
@@ -9,11 +10,16 @@ export class Exception extends Error {
       }
    }
 
+   context(context: any) {
+      this._context = context;
+      return this;
+   }
+
    toJSON() {
       return {
          error: this.message,
-         type: this.name
+         type: this.name,
-         //message: this.message
+         context: this._context
       };
    }
 }

app/src/core/events/Event.ts

@@ -1,17 +1,38 @@
-export abstract class Event<Params = any> {
+export type EventClass = {
+   new (params: any): Event<any, any>;
+   slug: string;
+};
+
+export abstract class Event<Params = any, Returning = void> {
+   _returning!: Returning;
+
    /**
     * Unique event slug
     * Must be static, because registering events is done by class
     */
    static slug: string = "untitled-event";
    params: Params;
+   returned: boolean = false;
+
+   validate(value: Returning): Event<Params, Returning> | void {
+      throw new EventReturnedWithoutValidation(this as any, value);
+   }
+
+   protected clone<This extends Event<Params, Returning> = Event<Params, Returning>>(
+      this: This,
+      params: Params
+   ): This {
+      const cloned = new (this.constructor as any)(params);
+      cloned.returned = true;
+      return cloned as This;
+   }
 
    constructor(params: Params) {
       this.params = params;
    }
 }
 
-// @todo: current workaround: potentially there is none and that's the way
+// @todo: current workaround: potentially there is "none" and that's the way
 export class NoParamEvent extends Event<null> {
    static override slug: string = "noparam-event";
 
@@ -19,3 +40,19 @@ export class NoParamEvent extends Event<null> {
       super(null);
    }
 }
+
+export class InvalidEventReturn extends Error {
+   constructor(expected: string, given: string) {
+      super(`Expected "${expected}", got "${given}"`);
+   }
+}
+
+export class EventReturnedWithoutValidation extends Error {
+   constructor(
+      event: EventClass,
+      public data: any
+   ) {
+      // @ts-expect-error slug is static
+      super(`Event "${event.constructor.slug}" returned without validation`);
+   }
+}

app/src/core/events/EventListener.ts

@@ -4,15 +4,16 @@ import type { EventClass } from "./EventManager";
 export const ListenerModes = ["sync", "async"] as const;
 export type ListenerMode = (typeof ListenerModes)[number];
 
-export type ListenerHandler<E extends Event = Event> = (
+export type ListenerHandler<E extends Event<any, any>> = (
    event: E,
-   slug: string,
+   slug: string
-) => Promise<void> | void;
+) => E extends Event<any, infer R> ? R | Promise<R | void> : never;
 
 export class EventListener<E extends Event = Event> {
    mode: ListenerMode = "async";
    event: EventClass;
    handler: ListenerHandler<E>;
+   once: boolean = false;
 
    constructor(event: EventClass, handler: ListenerHandler<E>, mode: ListenerMode = "async") {
       this.event = event;

app/src/core/events/EventManager.ts

@@ -1,31 +1,51 @@
-import type { Event } from "./Event";
+import { type Event, type EventClass, InvalidEventReturn } from "./Event";
 import { EventListener, type ListenerHandler, type ListenerMode } from "./EventListener";
 
+export type RegisterListenerConfig =
+   | ListenerMode
+   | {
+        mode?: ListenerMode;
+        once?: boolean;
+     };
+
 export interface EmitsEvents {
    emgr: EventManager;
 }
 
-export type EventClass = {
+// for compatibility, moved it to Event.ts
-   new (params: any): Event;
+export type { EventClass };
-   slug: string;
-};
 
 export class EventManager<
    RegisteredEvents extends Record<string, EventClass> = Record<string, EventClass>
 > {
    protected events: EventClass[] = [];
    protected listeners: EventListener[] = [];
+   enabled: boolean = true;
 
-   constructor(events?: RegisteredEvents, listeners?: EventListener[]) {
+   constructor(
+      events?: RegisteredEvents,
+      private options?: {
+         listeners?: EventListener[];
+         onError?: (event: Event, e: unknown) => void;
+         onInvalidReturn?: (event: Event, e: InvalidEventReturn) => void;
+         asyncExecutor?: typeof Promise.all;
+      }
+   ) {
       if (events) {
          this.registerEvents(events);
       }
 
-      if (listeners) {
+      options?.listeners?.forEach((l) => this.addListener(l));
-         for (const listener of listeners) {
+   }
-            this.addListener(listener);
+
-         }
+   enable() {
-      }
+      this.enabled = true;
+      return this;
+   }
+
+   disable() {
+      this.enabled = false;
+      return this;
    }
 
    clearEvents() {
@@ -39,6 +59,10 @@ export class EventManager<
       return this;
    }
 
+   getListeners(): EventListener[] {
+      return [...this.listeners];
+   }
+
    get Events(): { [K in keyof RegisteredEvents]: RegisteredEvents[K] } {
       // proxy class to access events
       return new Proxy(this, {
@@ -67,9 +91,11 @@ export class EventManager<
       return !!this.events.find((e) => slug === e.slug);
    }
 
-   protected throwIfEventNotRegistered(event: EventClass) {
+   protected throwIfEventNotRegistered(event: EventClass | Event | string) {
-      if (!this.eventExists(event)) {
+      if (!this.eventExists(event as any)) {
-         throw new Error(`Event "${event.slug}" not registered`);
+         // @ts-expect-error
+         const name = event.constructor?.slug ?? event.slug ?? event;
+         throw new Error(`Event "${name}" not registered`);
       }
    }
 
@@ -102,50 +128,108 @@ export class EventManager<
       return this;
    }
 
+   protected createEventListener(
+      _event: EventClass | string,
+      handler: ListenerHandler<any>,
+      _config: RegisterListenerConfig = "async"
+   ) {
+      const event =
+         typeof _event === "string" ? this.events.find((e) => e.slug === _event)! : _event;
+      const config = typeof _config === "string" ? { mode: _config } : _config;
+      const listener = new EventListener(event, handler, config.mode);
+      if (config.once) {
+         listener.once = true;
+      }
+      this.addListener(listener as any);
+   }
+
    onEvent<ActualEvent extends EventClass, Instance extends InstanceType<ActualEvent>>(
       event: ActualEvent,
       handler: ListenerHandler<Instance>,
-      mode: ListenerMode = "async"
+      config?: RegisterListenerConfig
    ) {
-      this.throwIfEventNotRegistered(event);
+      this.createEventListener(event, handler, config);
-
-      const listener = new EventListener(event, handler, mode);
-      this.addListener(listener as any);
    }
 
    on<Params = any>(
       slug: string,
       handler: ListenerHandler<Event<Params>>,
-      mode: ListenerMode = "async"
+      config?: RegisterListenerConfig
    ) {
-      const event = this.events.find((e) => e.slug === slug);
+      this.createEventListener(slug, handler, config);
-      if (!event) {
-         throw new Error(`Event "${slug}" not registered`);
-      }
-
-      this.onEvent(event, handler, mode);
    }
 
-   onAny(handler: ListenerHandler<Event<unknown>>, mode: ListenerMode = "async") {
+   onAny(handler: ListenerHandler<Event<unknown>>, config?: RegisterListenerConfig) {
-      this.events.forEach((event) => this.onEvent(event, handler, mode));
+      this.events.forEach((event) => this.onEvent(event, handler, config));
    }
 
-   async emit(event: Event) {
+   protected executeAsyncs(promises: (() => Promise<void>)[]) {
+      const executor = this.options?.asyncExecutor ?? ((e) => Promise.all(e));
+      executor(promises.map((p) => p())).then(() => void 0);
+   }
+
+   async emit<Actual extends Event<any, any>>(event: Actual): Promise<Actual> {
       // @ts-expect-error slug is static
       const slug = event.constructor.slug;
+      if (!this.enabled) {
+         console.log("EventManager disabled, not emitting", slug);
+         return event;
+      }
+
       if (!this.eventExists(event)) {
          throw new Error(`Event "${slug}" not registered`);
       }
 
-      const listeners = this.listeners.filter((listener) => listener.event.slug === slug);
+      const syncs: EventListener[] = [];
-      //console.log("---!-- emitting", slug, listeners.length);
+      const asyncs: (() => Promise<void>)[] = [];
+
+      this.listeners = this.listeners.filter((listener) => {
+         // if no match, keep and ignore
+         if (listener.event.slug !== slug) return true;
 
-      for (const listener of listeners) {
          if (listener.mode === "sync") {
-            await listener.handler(event, listener.event.slug);
+            syncs.push(listener);
          } else {
-            listener.handler(event, listener.event.slug);
+            asyncs.push(async () => await listener.handler(event, listener.event.slug));
+         }
+         // Remove if `once` is true, otherwise keep
+         return !listener.once;
+      });
+
+      // execute asyncs
+      this.executeAsyncs(asyncs);
+
+      // execute syncs
+      let _event: Actual = event;
+      for (const listener of syncs) {
+         try {
+            const return_value = (await listener.handler(_event, listener.event.slug)) as any;
+
+            if (typeof return_value !== "undefined") {
+               const newEvent = _event.validate(return_value);
+               // @ts-expect-error slug is static
+               if (newEvent && newEvent.constructor.slug === slug) {
+                  if (!newEvent.returned) {
+                     throw new Error(
+                        // @ts-expect-error slug is static
+                        `Returned event ${newEvent.constructor.slug} must be marked as returned.`
+                     );
+                  }
+                  _event = newEvent as Actual;
+               }
+            }
+         } catch (e) {
+            if (e instanceof InvalidEventReturn) {
+               this.options?.onInvalidReturn?.(_event, e);
+               console.warn(`Invalid return of event listener for "${slug}": ${e.message}`);
+            } else if (this.options?.onError) {
+               this.options.onError(_event, e);
+            } else {
+               throw e;
+            }
          }
       }
+
+      return _event;
    }
 }

app/src/core/events/index.ts

@@ -1,8 +1,8 @@
-export { Event, NoParamEvent } from "./Event";
+export { Event, NoParamEvent, InvalidEventReturn } from "./Event";
 export {
    EventListener,
    ListenerModes,
    type ListenerMode,
-   type ListenerHandler,
+   type ListenerHandler
 } from "./EventListener";
 export { EventManager, type EmitsEvents, type EventClass } from "./EventManager";

app/src/core/index.ts

@@ -3,7 +3,7 @@ import type { Hono, MiddlewareHandler } from "hono";
 export { tbValidator } from "./server/lib/tbValidator";
 export { Exception, BkndError } from "./errors";
 export { isDebug } from "./env";
-export { type PrimaryFieldType, config } from "./config";
+export { type PrimaryFieldType, config, type DB } from "./config";
 export { AwsClient } from "./clients/aws/AwsClient";
 export {
    SimpleRenderer,

app/src/core/object/SchemaObject.ts

@@ -1,10 +1,10 @@
-import { cloneDeep, get, has, mergeWith, omit, set } from "lodash-es";
+import { get, has, omit, set } from "lodash-es";
 import {
    Default,
    type Static,
    type TObject,
    getFullPathKeys,
-   mark,
+   mergeObjectWith,
    parse,
    stripMark
 } from "../utils";
@@ -33,7 +33,7 @@ export class SchemaObject<Schema extends TObject> {
    ) {
       this._default = Default(_schema, {} as any) as any;
       this._value = initial
-         ? parse(_schema, cloneDeep(initial as any), {
+         ? parse(_schema, structuredClone(initial as any), {
               forceParse: this.isForceParse(),
               skipMark: this.isForceParse()
            })
@@ -64,12 +64,17 @@ export class SchemaObject<Schema extends TObject> {
       return this._config;
    }
 
+   clone() {
+      return structuredClone(this._config);
+   }
+
    async set(config: Static<Schema>, noEmit?: boolean): Promise<Static<Schema>> {
-      const valid = parse(this._schema, cloneDeep(config) as any, {
+      const valid = parse(this._schema, structuredClone(config) as any, {
          forceParse: true,
          skipMark: this.isForceParse()
       });
-      const updatedConfig = noEmit ? valid : await this.onBeforeUpdate(this._config, valid);
+      // regardless of "noEmit" – this should always be triggered
+      const updatedConfig = await this.onBeforeUpdate(this._config, valid);
 
       this._value = updatedConfig;
       this._config = Object.freeze(updatedConfig);
@@ -113,7 +118,7 @@ export class SchemaObject<Schema extends TObject> {
    }
 
    async patch(path: string, value: any): Promise<[Partial<Static<Schema>>, Static<Schema>]> {
-      const current = cloneDeep(this._config);
+      const current = this.clone();
       const partial = path.length > 0 ? (set({}, path, value) as Partial<Static<Schema>>) : value;
 
       this.throwIfRestricted(partial);
@@ -121,15 +126,20 @@ export class SchemaObject<Schema extends TObject> {
 
       // overwrite arrays and primitives, only deep merge objects
       // @ts-ignore
-      const config = mergeWith(current, partial, (objValue, srcValue) => {
+      //console.log("---alt:new", _jsonp(mergeObject(current, partial)));
+      const config = mergeObjectWith(current, partial, (objValue, srcValue) => {
          if (Array.isArray(objValue) && Array.isArray(srcValue)) {
             return srcValue;
          }
       });
+      //console.log("---new", _jsonp(config));
 
       //console.log("overwritePaths", this.options?.overwritePaths);
       if (this.options?.overwritePaths) {
-         const keys = getFullPathKeys(value).map((k) => path + "." + k);
+         const keys = getFullPathKeys(value).map((k) => {
+            // only prepend path if given
+            return path.length > 0 ? path + "." + k : k;
+         });
          const overwritePaths = keys.filter((k) => {
             return this.options?.overwritePaths?.some((p) => {
                if (typeof p === "string") {
@@ -160,14 +170,14 @@ export class SchemaObject<Schema extends TObject> {
          }
       }
 
-      //console.log("patch", { path, value, partial, config, current });
+      //console.log("patch", _jsonp({ path, value, partial, config, current }));
 
       const newConfig = await this.set(config);
       return [partial, newConfig];
    }
 
    async overwrite(path: string, value: any): Promise<[Partial<Static<Schema>>, Static<Schema>]> {
-      const current = cloneDeep(this._config);
+      const current = this.clone();
       const partial = path.length > 0 ? (set({}, path, value) as Partial<Static<Schema>>) : value;
 
       this.throwIfRestricted(partial);
@@ -188,7 +198,7 @@ export class SchemaObject<Schema extends TObject> {
       if (p.length > 1) {
          const parent = p.slice(0, -1).join(".");
          if (!has(this._config, parent)) {
-            console.log("parent", parent, JSON.stringify(this._config, null, 2));
+            //console.log("parent", parent, JSON.stringify(this._config, null, 2));
             throw new Error(`Parent path "${parent}" does not exist`);
          }
       }
@@ -203,7 +213,7 @@ export class SchemaObject<Schema extends TObject> {
          throw new Error(`Path "${path}" does not exist`);
       }
 
-      const current = cloneDeep(this._config);
+      const current = this.clone();
       const removed = get(current, path) as Partial<Static<Schema>>;
       const config = omit(current, path);
       const newConfig = await this.set(config);

app/src/core/object/query/query.ts

@@ -49,7 +49,7 @@ type LiteralExpressionCondition<Exps extends Expressions> = {
    [key: string]: Primitive | ExpressionCondition<Exps>;
 };
 
-const OperandOr = "$or";
+const OperandOr = "$or" as const;
 type OperandCondition<Exps extends Expressions> = {
    [OperandOr]?: LiteralExpressionCondition<Exps> | ExpressionCondition<Exps>;
 };

app/src/core/registry/Registry.ts

@@ -1,29 +1,50 @@
 export type Constructor<T> = new (...args: any[]) => T;
-export class Registry<Item, Items extends Record<string, object> = Record<string, object>> {
+
+export type RegisterFn<Item> = (unknown: any) => Item;
+
+export class Registry<
+   Item,
+   Items extends Record<string, Item> = Record<string, Item>,
+   Fn extends RegisterFn<Item> = RegisterFn<Item>
+> {
    private is_set: boolean = false;
    private items: Items = {} as Items;
 
-   set<Actual extends Record<string, object>>(items: Actual) {
+   constructor(private registerFn?: Fn) {}
+
+   set<Actual extends Record<string, Item>>(items: Actual) {
       if (this.is_set) {
          throw new Error("Registry is already set");
       }
-      // @ts-ignore
+      this.items = items as unknown as Items;
-      this.items = items;
       this.is_set = true;
 
-      return this as unknown as Registry<Item, Actual>;
+      return this as unknown as Registry<Item, Actual, Fn>;
    }
 
    add(name: string, item: Item) {
-      // @ts-ignore
+      this.items[name as keyof Items] = item as Items[keyof Items];
-      this.items[name] = item;
       return this;
    }
 
+   register(name: string, specific: Parameters<Fn>[0]) {
+      if (this.registerFn) {
+         const item = this.registerFn(specific);
+         this.items[name as keyof Items] = item as Items[keyof Items];
+         return this;
+      }
+
+      return this.add(name, specific);
+   }
+
    get<Name extends keyof Items>(name: Name): Items[Name] {
       return this.items[name];
    }
 
+   has(name: keyof Items): boolean {
+      return name in this.items;
+   }
+
    all() {
       return this.items;
    }

app/src/core/server/flash.ts

@@ -4,14 +4,12 @@ import { setCookie } from "hono/cookie";
 const flash_key = "__bknd_flash";
 export type FlashMessageType = "error" | "warning" | "success" | "info";
 
-export async function addFlashMessage(
+export function addFlashMessage(c: Context, message: string, type: FlashMessageType = "info") {
-   c: Context,
+   if (c.req.header("Accept")?.includes("text/html")) {
-   message: string,
+      setCookie(c, flash_key, JSON.stringify({ type, message }), {
-   type: FlashMessageType = "info"
+         path: "/"
-) {
+      });
-   setCookie(c, flash_key, JSON.stringify({ type, message }), {
+   }
-      path: "/"
-   });
 }
 
 function getCookieValue(name) {

app/src/core/utils/DebugLogger.ts

@@ -20,11 +20,16 @@ export class DebugLogger {
       return this;
    }
 
+   reset() {
+      this.last = 0;
+      return this;
+   }
+
    log(...args: any[]) {
       if (!this._enabled) return this;
 
       const now = performance.now();
-      const time = Number.parseInt(String(now - this.last));
+      const time = this.last === 0 ? 0 : Number.parseInt(String(now - this.last));
       const indents = "  ".repeat(this._context.length);
       const context =
          this._context.length > 0 ? `[${this._context[this._context.length - 1]}]` : "";

app/src/core/utils/index.ts

@@ -11,3 +11,5 @@ export * from "./crypto";
 export * from "./uuid";
 export { FromSchema } from "./typebox/from-schema";
 export * from "./test";
+export * from "./runtime";
+export * from "./numbers";

app/src/core/utils/numbers.ts

@@ -0,0 +1,5 @@
+export function clampNumber(value: number, min: number, max: number): number {
+   const lower = Math.min(min, max);
+   const upper = Math.max(min, max);
+   return Math.max(lower, Math.min(value, upper));
+}

app/src/core/utils/objects.ts

@@ -4,6 +4,14 @@ export function _jsonp(obj: any, space = 2): string {
    return JSON.stringify(obj, null, space);
 }
 
+export function isPlainObject(value: unknown): value is Record<string, unknown> {
+   return Object.prototype.toString.call(value) === "[object Object]";
+}
+
+export function isObject(value: unknown): value is Record<string, unknown> {
+   return value !== null && typeof value === "object";
+}
+
 export function safelyParseObjectValues<T extends { [key: string]: any }>(obj: T): T {
    return Object.entries(obj).reduce((acc, [key, value]) => {
       try {
@@ -97,15 +105,6 @@ export function objectEach<T extends Record<string, any>, U>(
    );
 }
 
-/**
- * Simple object check.
- * @param item
- * @returns {boolean}
- */
-export function isObject(item) {
-   return item && typeof item === "object" && !Array.isArray(item);
-}
-
 /**
  * Deep merge two objects.
  * @param target
@@ -197,3 +196,73 @@ export function objectCleanEmpty<Obj extends { [key: string]: any }>(obj: Obj):
       return acc;
    }, {} as any);
 }
+
+/**
+ * Lodash's merge implementation caused issues in Next.js environments
+ * From: https://thescottyjam.github.io/snap.js/#!/nolodash/merge
+ * NOTE: This mutates `object`. It also may mutate anything that gets attached to `object` during the merge.
+ * @param object
+ * @param sources
+ */
+export function mergeObject(object, ...sources) {
+   for (const source of sources) {
+      for (const [key, value] of Object.entries(source)) {
+         if (value === undefined) {
+            continue;
+         }
+
+         // These checks are a week attempt at mimicking the various edge-case behaviors
+         // that Lodash's `_.merge()` exhibits. Feel free to simplify and
+         // remove checks that you don't need.
+         if (!isPlainObject(value) && !Array.isArray(value)) {
+            object[key] = value;
+         } else if (Array.isArray(value) && !Array.isArray(object[key])) {
+            object[key] = value;
+         } else if (!isObject(object[key])) {
+            object[key] = value;
+         } else {
+            mergeObject(object[key], value);
+         }
+      }
+   }
+
+   return object;
+}
+
+/**
+ * Lodash's mergeWith implementation caused issues in Next.js environments
+ * From: https://thescottyjam.github.io/snap.js/#!/nolodash/mergeWith
+ * NOTE: This mutates `object`. It also may mutate anything that gets attached to `object` during the merge.
+ * @param object
+ * @param sources
+ * @param customizer
+ */
+export function mergeObjectWith(object, source, customizer) {
+   for (const [key, value] of Object.entries(source)) {
+      const mergedValue = customizer(object[key], value, key, object, source);
+      if (mergedValue !== undefined) {
+         object[key] = mergedValue;
+         continue;
+      }
+      // Otherwise, fall back to default behavior
+
+      if (value === undefined) {
+         continue;
+      }
+
+      // These checks are a week attempt at mimicking the various edge-case behaviors
+      // that Lodash's `_.merge()` exhibits. Feel free to simplify and
+      // remove checks that you don't need.
+      if (!isPlainObject(value) && !Array.isArray(value)) {
+         object[key] = value;
+      } else if (Array.isArray(value) && !Array.isArray(object[key])) {
+         object[key] = value;
+      } else if (!isObject(object[key])) {
+         object[key] = value;
+      } else {
+         mergeObjectWith(object[key], value, customizer);
+      }
+   }
+
+   return object;
+}

app/src/core/utils/runtime.ts

@@ -0,0 +1,41 @@
+import { getRuntimeKey as honoGetRuntimeKey } from "hono/adapter";
+
+/**
+ * Adds additional checks for nextjs
+ */
+export function getRuntimeKey(): string {
+   const global = globalThis as any;
+
+   // Detect Next.js server-side runtime
+   if (global?.process?.env?.NEXT_RUNTIME === "nodejs") {
+      return "nextjs";
+   }
+
+   // Detect Next.js edge runtime
+   if (global?.process?.env?.NEXT_RUNTIME === "edge") {
+      return "nextjs-edge";
+   }
+
+   // Detect Next.js client-side runtime
+   // @ts-ignore
+   if (typeof window !== "undefined" && window.__NEXT_DATA__) {
+      return "nextjs-client";
+   }
+
+   return honoGetRuntimeKey();
+}
+
+const features = {
+   // supports the redirect of not full qualified addresses
+   // not supported in nextjs
+   redirects_non_fq: true
+};
+
+export function runtimeSupports(feature: keyof typeof features) {
+   const runtime = getRuntimeKey();
+   if (runtime.startsWith("nextjs")) {
+      features.redirects_non_fq = false;
+   }
+
+   return features[feature];
+}

app/src/core/utils/strings.ts

@@ -104,3 +104,14 @@ export function replaceSimplePlaceholders(str: string, vars: Record<string, any>
       return key in vars ? vars[key] : match;
    });
 }
+
+export function patternMatch(target: string, pattern: RegExp | string): boolean {
+   if (pattern instanceof RegExp) {
+      return pattern.test(target);
+   } else if (typeof pattern === "string" && pattern.startsWith("/")) {
+      return new RegExp(pattern).test(target);
+   } else if (typeof pattern === "string") {
+      return target.startsWith(pattern);
+   }
+   return false;
+}

app/src/core/utils/test.ts

@@ -7,15 +7,30 @@ const _oldConsoles = {
 
 export async function withDisabledConsole<R>(
    fn: () => Promise<R>,
-   severities: ConsoleSeverity[] = ["log"]
+   severities: ConsoleSeverity[] = ["log", "warn", "error"]
 ): Promise<R> {
-   const enable = disableConsoleLog(severities);
+   const _oldConsoles = {
-   const result = await fn();
+      log: console.log,
-   enable();
+      warn: console.warn,
-   return result;
+      error: console.error
+   };
+   disableConsoleLog(severities);
+   const enable = () => {
+      Object.entries(_oldConsoles).forEach(([severity, fn]) => {
+         console[severity as ConsoleSeverity] = fn;
+      });
+   };
+   try {
+      const result = await fn();
+      enable();
+      return result;
+   } catch (e) {
+      enable();
+      throw e;
+   }
 }
 
-export function disableConsoleLog(severities: ConsoleSeverity[] = ["log"]) {
+export function disableConsoleLog(severities: ConsoleSeverity[] = ["log", "warn"]) {
    severities.forEach((severity) => {
       console[severity] = () => null;
    });

app/src/core/utils/typebox/index.ts

@@ -115,6 +115,7 @@ export function parse<Schema extends TSchema = TSchema>(
    } else if (options?.onError) {
       options.onError(Errors(schema, data));
    } else {
+      //console.warn("errors", JSON.stringify([...Errors(schema, data)], null, 2));
       throw new TypeInvalidError(schema, data);
    }
 

app/src/data/AppData.ts

@@ -1,66 +1,42 @@
 import { transformObject } from "core/utils";
-import { DataPermissions, Entity, EntityIndex, type EntityManager, type Field } from "data";
+import {
+   DataPermissions,
+   type Entity,
+   EntityIndex,
+   type EntityManager,
+   constructEntity,
+   constructRelation
+} from "data";
 import { Module } from "modules/Module";
 import { DataController } from "./api/DataController";
-import {
+import { type AppDataConfig, dataConfigSchema } from "./data-schema";
-   type AppDataConfig,
-   FIELDS,
-   RELATIONS,
-   type TAppDataEntity,
-   type TAppDataRelation,
-   dataConfigSchema
-} from "./data-schema";
-
-export class AppData<DB> extends Module<typeof dataConfigSchema> {
-   static constructEntity(name: string, entityConfig: TAppDataEntity) {
-      const fields = transformObject(entityConfig.fields ?? {}, (fieldConfig, name) => {
-         const { type } = fieldConfig;
-         if (!(type in FIELDS)) {
-            throw new Error(`Field type "${type}" not found`);
-         }
-
-         const { field } = FIELDS[type as any];
-         const returnal = new field(name, fieldConfig.config) as Field;
-         return returnal;
-      });
-
-      // @todo: entity must be migrated to typebox
-      return new Entity(
-         name,
-         Object.values(fields),
-         entityConfig.config as any,
-         entityConfig.type as any
-      );
-   }
-
-   static constructRelation(
-      relationConfig: TAppDataRelation,
-      resolver: (name: Entity | string) => Entity
-   ) {
-      return new RELATIONS[relationConfig.type].cls(
-         resolver(relationConfig.source),
-         resolver(relationConfig.target),
-         relationConfig.config
-      );
-   }
 
+export class AppData extends Module<typeof dataConfigSchema> {
    override async build() {
-      const entities = transformObject(this.config.entities ?? {}, (entityConfig, name) => {
+      const {
-         return AppData.constructEntity(name, entityConfig);
+         entities: _entities = {},
+         relations: _relations = {},
+         indices: _indices = {}
+      } = this.config;
+
+      this.ctx.logger.context("AppData").log("building with entities", Object.keys(_entities));
+
+      const entities = transformObject(_entities, (entityConfig, name) => {
+         return constructEntity(name, entityConfig);
       });
 
       const _entity = (_e: Entity | string): Entity => {
          const name = typeof _e === "string" ? _e : _e.name;
          const entity = entities[name];
          if (entity) return entity;
-         throw new Error(`Entity "${name}" not found`);
+         throw new Error(`[AppData] Entity "${name}" not found`);
       };
 
-      const relations = transformObject(this.config.relations ?? {}, (relation) =>
+      const relations = transformObject(_relations, (relation) =>
-         AppData.constructRelation(relation, _entity)
+         constructRelation(relation, _entity)
       );
 
-      const indices = transformObject(this.config.indices ?? {}, (index, name) => {
+      const indices = transformObject(_indices, (index, name) => {
          const entity = _entity(index.entity)!;
          const fields = index.fields.map((f) => entity.field(f)!);
          return new EntityIndex(entity, fields, index.unique, name);
@@ -84,6 +60,7 @@ export class AppData<DB> extends Module<typeof dataConfigSchema> {
       );
       this.ctx.guard.registerPermissions(Object.values(DataPermissions));
 
+      this.ctx.logger.clear();
       this.setBuilt();
    }
 
@@ -91,7 +68,7 @@ export class AppData<DB> extends Module<typeof dataConfigSchema> {
       return dataConfigSchema;
    }
 
-   get em(): EntityManager<DB> {
+   get em(): EntityManager {
       this.throwIfNotBuilt();
       return this.ctx.em;
    }
@@ -101,18 +78,9 @@ export class AppData<DB> extends Module<typeof dataConfigSchema> {
    }
 
    override getOverwritePaths() {
-      return [
+      return [/^entities\..*\.config$/, /^entities\..*\.fields\..*\.config$/];
-         /^entities\..*\.config$/,
-         /^entities\..*\.fields\..*\.config$/
-         ///^entities\..*\.fields\..*\.config\.schema$/
-      ];
    }
 
-   /*registerController(server: AppServer) {
-      console.log("adding data controller to", this.basepath);
-      server.add(this.basepath, new DataController(this.em));
-   }*/
-
    override toJSON(secrets?: boolean): AppDataConfig {
       return {
          ...this.config,

app/src/data/api/DataApi.ts

@@ -1,62 +1,83 @@
-import type { EntityData, RepoQuery, RepositoryResponse } from "data";
+import type { DB } from "core";
+import type { EntityData, RepoQuery, RepoQueryIn, RepositoryResponse } from "data";
 import { type BaseModuleApiOptions, ModuleApi, type PrimaryFieldType } from "modules";
 
 export type DataApiOptions = BaseModuleApiOptions & {
-   defaultQuery?: Partial<RepoQuery>;
+   queryLengthLimit: number;
+   defaultQuery: Partial<RepoQuery>;
 };
 
 export class DataApi extends ModuleApi<DataApiOptions> {
    protected override getDefaultOptions(): Partial<DataApiOptions> {
       return {
          basepath: "/api/data",
+         queryLengthLimit: 1000,
          defaultQuery: {
             limit: 10
          }
       };
    }
 
-   async readOne(
+   readOne<E extends keyof DB | string, Data = E extends keyof DB ? DB[E] : EntityData>(
-      entity: string,
+      entity: E,
       id: PrimaryFieldType,
-      query: Partial<Omit<RepoQuery, "where" | "limit" | "offset">> = {}
+      query: Omit<RepoQueryIn, "where" | "limit" | "offset"> = {}
    ) {
-      return this.get<RepositoryResponse<EntityData>>([entity, id], query);
+      return this.get<Pick<RepositoryResponse<Data>, "meta" | "data">>([entity as any, id], query);
    }
 
-   async readMany(entity: string, query: Partial<RepoQuery> = {}) {
+   readMany<E extends keyof DB | string, Data = E extends keyof DB ? DB[E] : EntityData>(
-      return this.get<Pick<RepositoryResponse, "meta" | "data">>(
+      entity: E,
-         [entity],
+      query: RepoQueryIn = {}
+   ) {
+      type T = Pick<RepositoryResponse<Data[]>, "meta" | "data">;
+
+      const input = query ?? this.options.defaultQuery;
+      const req = this.get<T>([entity as any], input);
+
+      if (req.request.url.length <= this.options.queryLengthLimit) {
+         return req;
+      }
+
+      return this.post<T>([entity as any, "query"], input);
+   }
+
+   readManyByReference<
+      E extends keyof DB | string,
+      R extends keyof DB | string,
+      Data = R extends keyof DB ? DB[R] : EntityData
+   >(entity: E, id: PrimaryFieldType, reference: R, query: RepoQueryIn = {}) {
+      return this.get<Pick<RepositoryResponse<Data[]>, "meta" | "data">>(
+         [entity as any, id, reference],
          query ?? this.options.defaultQuery
       );
    }
 
-   async readManyByReference(
+   createOne<E extends keyof DB | string, Data = E extends keyof DB ? DB[E] : EntityData>(
-      entity: string,
+      entity: E,
-      id: PrimaryFieldType,
+      input: Omit<Data, "id">
-      reference: string,
-      query: Partial<RepoQuery> = {}
    ) {
-      return this.get<Pick<RepositoryResponse, "meta" | "data">>(
+      return this.post<RepositoryResponse<Data>>([entity as any], input);
-         [entity, id, reference],
-         query ?? this.options.defaultQuery
-      );
    }
 
-   async createOne(entity: string, input: EntityData) {
+   updateOne<E extends keyof DB | string, Data = E extends keyof DB ? DB[E] : EntityData>(
-      return this.post<RepositoryResponse<EntityData>>([entity], input);
+      entity: E,
+      id: PrimaryFieldType,
+      input: Partial<Omit<Data, "id">>
+   ) {
+      return this.patch<RepositoryResponse<Data>>([entity as any, id], input);
    }
 
-   async updateOne(entity: string, id: PrimaryFieldType, input: EntityData) {
+   deleteOne<E extends keyof DB | string, Data = E extends keyof DB ? DB[E] : EntityData>(
-      return this.patch<RepositoryResponse<EntityData>>([entity, id], input);
+      entity: E,
+      id: PrimaryFieldType
+   ) {
+      return this.delete<RepositoryResponse<Data>>([entity as any, id]);
    }
 
-   async deleteOne(entity: string, id: PrimaryFieldType) {
+   count<E extends keyof DB | string>(entity: E, where: RepoQuery["where"] = {}) {
-      return this.delete<RepositoryResponse<EntityData>>([entity, id]);
+      return this.post<RepositoryResponse<{ entity: E; count: number }>>(
-   }
+         [entity as any, "fn", "count"],
-
-   async count(entity: string, where: RepoQuery["where"] = {}) {
-      return this.post<RepositoryResponse<{ entity: string; count: number }>>(
-         [entity, "fn", "count"],
          where
       );
    }

app/src/data/api/DataController.ts

@@ -1,32 +1,26 @@
-import { type ClassController, isDebug, tbValidator as tb } from "core";
+import { isDebug, tbValidator as tb } from "core";
-import { Type, objectCleanEmpty, objectTransform } from "core/utils";
+import { StringEnum, Type } from "core/utils";
 import {
    DataPermissions,
    type EntityData,
    type EntityManager,
-   FieldClassMap,
    type MutatorResponse,
-   PrimaryField,
    type RepoQuery,
    type RepositoryResponse,
-   TextField,
    querySchema
 } from "data";
-import { Hono } from "hono";
 import type { Handler } from "hono/types";
 import type { ModuleBuildContext } from "modules";
+import { Controller } from "modules/Controller";
 import * as SystemPermissions from "modules/permissions";
-import { type AppDataConfig, FIELDS } from "../data-schema";
+import type { AppDataConfig } from "../data-schema";
 
-export class DataController implements ClassController {
+export class DataController extends Controller {
    constructor(
       private readonly ctx: ModuleBuildContext,
       private readonly config: AppDataConfig
    ) {
-      /*console.log(
+      super();
-         "data controller",
-         this.em.entities.map((e) => e.name)
-      );*/
    }
 
    get em(): EntityManager<any> {
@@ -74,8 +68,10 @@ export class DataController implements ClassController {
       }
    }
 
-   getController(): Hono<any> {
+   override getController() {
-      const hono = new Hono();
+      const { permission, auth } = this.middlewares;
+      const hono = this.create().use(auth(), permission(SystemPermissions.accessApi));
+
       const definedEntities = this.em.entities.map((e) => e.name);
       const tbNumber = Type.Transform(Type.String({ pattern: "^[1-9][0-9]{0,}$" }))
          .Decode(Number.parseInt)
@@ -89,11 +85,6 @@ export class DataController implements ClassController {
          return func;
       }
 
-      hono.use("*", async (c, next) => {
-         this.ctx.guard.throwUnlessGranted(SystemPermissions.accessApi);
-         await next();
-      });
-
       // info
       hono.get(
          "/",
@@ -104,9 +95,7 @@ export class DataController implements ClassController {
       );
 
       // sync endpoint
-      hono.get("/sync", async (c) => {
+      hono.get("/sync", permission(DataPermissions.databaseSync), async (c) => {
-         this.guard.throwUnlessGranted(DataPermissions.databaseSync);
-
          const force = c.req.query("force") === "1";
          const drop = c.req.query("drop") === "1";
          //console.log("force", force);
@@ -126,10 +115,9 @@ export class DataController implements ClassController {
          // fn: count
          .post(
             "/:entity/fn/count",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity } = c.req.valid("param");
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -143,10 +131,9 @@ export class DataController implements ClassController {
          // fn: exists
          .post(
             "/:entity/fn/exists",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity } = c.req.valid("param");
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -163,15 +150,13 @@ export class DataController implements ClassController {
        */
       hono
          // read entity schema
-         .get("/schema.json", async (c) => {
+         .get("/schema.json", permission(DataPermissions.entityRead), async (c) => {
-            this.guard.throwUnlessGranted(DataPermissions.entityRead);
+            const $id = `${this.config.basepath}/schema.json`;
-            const url = new URL(c.req.url);
-            const $id = `${url.origin}${this.config.basepath}/schema.json`;
             const schemas = Object.fromEntries(
                this.em.entities.map((e) => [
                   e.name,
                   {
-                     $ref: `schemas/${e.name}`
+                     $ref: `${this.config.basepath}/schemas/${e.name}`
                   }
                ])
             );
@@ -183,22 +168,27 @@ export class DataController implements ClassController {
          })
          // read schema
          .get(
-            "/schemas/:entity",
+            "/schemas/:entity/:context?",
-            tb("param", Type.Object({ entity: Type.String() })),
+            permission(DataPermissions.entityRead),
+            tb(
+               "param",
+               Type.Object({
+                  entity: Type.String(),
+                  context: Type.Optional(StringEnum(["create", "update"]))
+               })
+            ),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                //console.log("request", c.req.raw);
-               const { entity } = c.req.param();
+               const { entity, context } = c.req.param();
                if (!this.entityExists(entity)) {
                   console.log("not found", entity, definedEntities);
                   return c.notFound();
                }
                const _entity = this.em.entity(entity);
-               const schema = _entity.toSchema();
+               const schema = _entity.toSchema({ context } as any);
                const url = new URL(c.req.url);
                const base = `${url.origin}${this.config.basepath}`;
-               const $id = `${base}/schemas/${entity}`;
+               const $id = `${this.config.basepath}/schemas/${entity}`;
                return c.json({
                   $schema: `${base}/schema.json`,
                   $id,
@@ -211,11 +201,10 @@ export class DataController implements ClassController {
          // read many
          .get(
             "/:entity",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             tb("query", querySchema),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                //console.log("request", c.req.raw);
                const { entity } = c.req.param();
                if (!this.entityExists(entity)) {
@@ -233,6 +222,7 @@ export class DataController implements ClassController {
          // read one
          .get(
             "/:entity/:id",
+            permission(DataPermissions.entityRead),
             tb(
                "param",
                Type.Object({
@@ -241,11 +231,7 @@ export class DataController implements ClassController {
                })
             ),
             tb("query", querySchema),
-            /*zValidator("param", z.object({ entity: z.string(), id: z.coerce.number() })),
-            zValidator("query", repoQuerySchema),*/
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity, id } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -259,6 +245,7 @@ export class DataController implements ClassController {
          // read many by reference
          .get(
             "/:entity/:id/:reference",
+            permission(DataPermissions.entityRead),
             tb(
                "param",
                Type.Object({
@@ -269,8 +256,6 @@ export class DataController implements ClassController {
             ),
             tb("query", querySchema),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity, id, reference } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -287,17 +272,16 @@ export class DataController implements ClassController {
          // func query
          .post(
             "/:entity/query",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             tb("json", querySchema),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
                }
                const options = (await c.req.valid("json")) as RepoQuery;
-               console.log("options", options);
+               //console.log("options", options);
                const result = await this.em.repository(entity).findMany(options);
 
                return c.json(this.repoResult(result), { status: result.data ? 200 : 404 });
@@ -309,25 +293,27 @@ export class DataController implements ClassController {
        */
       // insert one
       hono
-         .post("/:entity", tb("param", Type.Object({ entity: Type.String() })), async (c) => {
+         .post(
-            this.guard.throwUnlessGranted(DataPermissions.entityCreate);
+            "/:entity",
+            permission(DataPermissions.entityCreate),
+            tb("param", Type.Object({ entity: Type.String() })),
+            async (c) => {
+               const { entity } = c.req.param();
+               if (!this.entityExists(entity)) {
+                  return c.notFound();
+               }
+               const body = (await c.req.json()) as EntityData;
+               const result = await this.em.mutator(entity).insertOne(body);
 
-            const { entity } = c.req.param();
+               return c.json(this.mutatorResult(result), 201);
-            if (!this.entityExists(entity)) {
-               return c.notFound();
             }
-            const body = (await c.req.json()) as EntityData;
+         )
-            const result = await this.em.mutator(entity).insertOne(body);
-
-            return c.json(this.mutatorResult(result), 201);
-         })
          // update one
          .patch(
             "/:entity/:id",
+            permission(DataPermissions.entityUpdate),
             tb("param", Type.Object({ entity: Type.String(), id: tbNumber })),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityUpdate);
-
                const { entity, id } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -341,6 +327,8 @@ export class DataController implements ClassController {
          // delete one
          .delete(
             "/:entity/:id",
+
+            permission(DataPermissions.entityDelete),
             tb("param", Type.Object({ entity: Type.String(), id: tbNumber })),
             async (c) => {
                this.guard.throwUnlessGranted(DataPermissions.entityDelete);
@@ -358,11 +346,10 @@ export class DataController implements ClassController {
          // delete many
          .delete(
             "/:entity",
+            permission(DataPermissions.entityDelete),
             tb("param", Type.Object({ entity: Type.String() })),
             tb("json", querySchema.properties.where),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityDelete);
-
                //console.log("request", c.req.raw);
                const { entity } = c.req.param();
                if (!this.entityExists(entity)) {

app/src/data/connection/DummyConnection.ts

@@ -0,0 +1,7 @@
+import { Connection } from "./Connection";
+
+export class DummyConnection extends Connection {
+   constructor() {
+      super(undefined as any);
+   }
+}

app/src/data/connection/SqliteLocalConnection.ts

@@ -1,6 +1,5 @@
-import type { DatabaseIntrospector, SqliteDatabase } from "kysely";
+import { type DatabaseIntrospector, ParseJSONResultsPlugin, type SqliteDatabase } from "kysely";
 import { Kysely, SqliteDialect } from "kysely";
-import { DeserializeJsonValuesPlugin } from "../plugins/DeserializeJsonValuesPlugin";
 import { SqliteConnection } from "./SqliteConnection";
 import { SqliteIntrospector } from "./SqliteIntrospector";
 
@@ -14,7 +13,7 @@ class CustomSqliteDialect extends SqliteDialect {
 
 export class SqliteLocalConnection extends SqliteConnection {
    constructor(private database: SqliteDatabase) {
-      const plugins = [new DeserializeJsonValuesPlugin()];
+      const plugins = [new ParseJSONResultsPlugin()];
       const kysely = new Kysely({
          dialect: new CustomSqliteDialect({ database }),
          plugins

app/src/data/entities/Entity.ts

@@ -98,8 +98,8 @@ export class Entity<
 
    getDefaultSort() {
       return {
-         by: this.config.sort_field,
+         by: this.config.sort_field ?? "id",
-         dir: this.config.sort_dir
+         dir: this.config.sort_dir ?? "asc"
       };
    }
 
@@ -140,7 +140,7 @@ export class Entity<
       return this.fields.find((field) => field.name === name);
    }
 
-   __experimental_replaceField(name: string, field: Field) {
+   __replaceField(name: string, field: Field) {
       const index = this.fields.findIndex((f) => f.name === name);
       if (index === -1) {
          throw new Error(`Field "${name}" not found on entity "${this.name}"`);
@@ -158,7 +158,7 @@ export class Entity<
    }
 
    get label(): string {
-      return snakeToPascalWithSpaces(this.config.name ?? this.name);
+      return this.config.name ?? snakeToPascalWithSpaces(this.name);
    }
 
    field(name: string): Field | undefined {
@@ -192,14 +192,41 @@ export class Entity<
       this.data = data;
    }
 
-   isValidData(data: EntityData, context: TActionContext, explain?: boolean): boolean {
+   // @todo: add tests
+   isValidData(
+      data: EntityData,
+      context: TActionContext,
+      options?: {
+         explain?: boolean;
+         ignoreUnknown?: boolean;
+      }
+   ): boolean {
+      if (typeof data !== "object") {
+         if (options?.explain) {
+            throw new Error(`Entity "${this.name}" data must be an object`);
+         }
+      }
+
       const fields = this.getFillableFields(context, false);
-      //const fields = this.fields;
+
-      //console.log("data", data);
+      if (options?.ignoreUnknown !== true) {
+         const field_names = fields.map((f) => f.name);
+         const given_keys = Object.keys(data);
+         const unknown_keys = given_keys.filter((key) => !field_names.includes(key));
+
+         if (unknown_keys.length > 0) {
+            if (options?.explain) {
+               throw new Error(
+                  `Entity "${this.name}" data must only contain known keys, unknown: "${unknown_keys}"`
+               );
+            }
+         }
+      }
+
       for (const field of fields) {
          if (!field.isValid(data[field.name], context)) {
             console.log("Entity.isValidData:invalid", context, field.name, data[field.name]);
-            if (explain) {
+            if (options?.explain) {
                throw new Error(`Field "${field.name}" has invalid data: "${data[field.name]}"`);
             }
 
@@ -210,20 +237,34 @@ export class Entity<
       return true;
    }
 
-   toSchema(clean?: boolean): object {
+   toSchema(options?: { clean: boolean; context?: "create" | "update" }): object {
-      const fields = Object.fromEntries(this.fields.map((field) => [field.name, field]));
+      let fields: Field[];
+      switch (options?.context) {
+         case "create":
+         case "update":
+            fields = this.getFillableFields(options.context);
+            break;
+         default:
+            fields = this.getFields(true);
+      }
+
+      const _fields = Object.fromEntries(fields.map((field) => [field.name, field]));
       const schema = Type.Object(
-         transformObject(fields, (field) => ({
+         transformObject(_fields, (field) => {
-            title: field.config.label,
+            //const hidden = field.isHidden(options?.context);
-            $comment: field.config.description,
+            const fillable = field.isFillable(options?.context);
-            $field: field.type,
+            return {
-            readOnly: !field.isFillable("update") ? true : undefined,
+               title: field.config.label,
-            writeOnly: !field.isFillable("create") ? true : undefined,
+               $comment: field.config.description,
-            ...field.toJsonSchema()
+               $field: field.type,
-         }))
+               readOnly: !fillable ? true : undefined,
+               ...field.toJsonSchema()
+            };
+         }),
+         { additionalProperties: false }
       );
 
-      return clean ? JSON.parse(JSON.stringify(schema)) : schema;
+      return options?.clean ? JSON.parse(JSON.stringify(schema)) : schema;
    }
 
    toJSON() {

app/src/data/entities/EntityManager.ts

@@ -1,3 +1,4 @@
+import type { DB as DefaultDB } from "core";
 import { EventManager } from "core/events";
 import { sql } from "kysely";
 import { Connection } from "../connection/Connection";
@@ -14,7 +15,18 @@ import { SchemaManager } from "../schema/SchemaManager";
 import { Entity } from "./Entity";
 import { type EntityData, Mutator, Repository } from "./index";
 
-export class EntityManager<DB> {
+type EntitySchema<
+   TBD extends object = DefaultDB,
+   E extends Entity | keyof TBD | string = string
+> = E extends Entity<infer Name>
+   ? Name extends keyof TBD
+      ? Name
+      : never
+   : E extends keyof TBD
+     ? E
+     : never;
+
+export class EntityManager<TBD extends object = DefaultDB> {
    connection: Connection;
 
    private _entities: Entity[] = [];
@@ -50,7 +62,7 @@ export class EntityManager<DB> {
     * Forks the EntityManager without the EventManager.
     * This is useful when used inside an event handler.
     */
-   fork(): EntityManager<DB> {
+   fork(): EntityManager {
       return new EntityManager(this._entities, this.connection, this._relations, this._indices);
    }
 
@@ -87,10 +99,30 @@ export class EntityManager<DB> {
       this.entities.push(entity);
    }
 
-   entity(name: string): Entity {
+   __replaceEntity(entity: Entity, name: string | undefined = entity.name) {
-      const entity = this.entities.find((e) => e.name === name);
+      const entityIndex = this._entities.findIndex((e) => e.name === name);
+
+      if (entityIndex === -1) {
+         throw new Error(`Entity "${name}" not found and cannot be replaced`);
+      }
+
+      this._entities[entityIndex] = entity;
+
+      // caused issues because this.entity() was using a reference (for when initial config was given)
+   }
+
+   entity<Silent extends true | false = false>(
+      e: Entity | keyof TBD | string,
+      silent?: Silent
+   ): Silent extends true ? Entity | undefined : Entity {
+      // make sure to always retrieve by name
+      const entity = this.entities.find((entity) =>
+         e instanceof Entity ? entity.name === e.name : entity.name === e
+      );
+
       if (!entity) {
-         throw new EntityNotDefinedException(name);
+         if (silent === true) return undefined as any;
+         throw new EntityNotDefinedException(e instanceof Entity ? e.name : (e as string));
       }
 
       return entity;
@@ -162,28 +194,18 @@ export class EntityManager<DB> {
       return this.relations.relationReferencesOf(this.entity(entity_name));
    }
 
-   repository(_entity: Entity | string) {
+   repository<E extends Entity | keyof TBD | string>(
-      const entity = _entity instanceof Entity ? _entity : this.entity(_entity);
+      entity: E
-      return new Repository(this, entity, this.emgr);
+   ): Repository<TBD, EntitySchema<TBD, E>> {
+      return this.repo(entity);
    }
 
-   repo<E extends Entity>(
+   repo<E extends Entity | keyof TBD | string>(entity: E): Repository<TBD, EntitySchema<TBD, E>> {
-      _entity: E
+      return new Repository(this, this.entity(entity), this.emgr);
-   ): Repository<
-      DB,
-      E extends Entity<infer Name> ? (Name extends keyof DB ? Name : never) : never
-   > {
-      return new Repository(this, _entity, this.emgr);
    }
 
-   _repo<TB extends keyof DB>(_entity: TB): Repository<DB, TB> {
+   mutator<E extends Entity | keyof TBD | string>(entity: E): Mutator<TBD, EntitySchema<TBD, E>> {
-      const entity = this.entity(_entity as any);
+      return new Mutator(this, this.entity(entity), this.emgr);
-      return new Repository(this, entity, this.emgr);
-   }
-
-   mutator(_entity: Entity | string) {
-      const entity = _entity instanceof Entity ? _entity : this.entity(_entity);
-      return new Mutator(this, entity, this.emgr);
    }
 
    addIndex(index: EntityIndex, force = false) {

app/src/data/entities/Mutator.ts

@@ -1,4 +1,4 @@
-import type { PrimaryFieldType } from "core";
+import type { DB as DefaultDB, PrimaryFieldType } from "core";
 import { type EmitsEvents, EventManager } from "core/events";
 import type { DeleteQueryBuilder, InsertQueryBuilder, UpdateQueryBuilder } from "kysely";
 import { type TActionContext, WhereBuilder } from "..";
@@ -25,8 +25,14 @@ export type MutatorResponse<T = EntityData[]> = {
    data: T;
 };
 
-export class Mutator<DB> implements EmitsEvents {
+export class Mutator<
-   em: EntityManager<DB>;
+   TBD extends object = DefaultDB,
+   TB extends keyof TBD = any,
+   Output = TBD[TB],
+   Input = Omit<Output, "id">
+> implements EmitsEvents
+{
+   em: EntityManager<TBD>;
    entity: Entity;
    static readonly Events = MutatorEvents;
    emgr: EventManager<typeof MutatorEvents>;
@@ -37,7 +43,7 @@ export class Mutator<DB> implements EmitsEvents {
       this.__unstable_disable_system_entity_creation = value;
    }
 
-   constructor(em: EntityManager<DB>, entity: Entity, emgr?: EventManager<any>) {
+   constructor(em: EntityManager<TBD>, entity: Entity, emgr?: EventManager<any>) {
       this.em = em;
       this.entity = entity;
       this.emgr = emgr ?? new EventManager(MutatorEvents);
@@ -47,13 +53,13 @@ export class Mutator<DB> implements EmitsEvents {
       return this.em.connection.kysely;
    }
 
-   async getValidatedData(data: EntityData, context: TActionContext): Promise<EntityData> {
+   async getValidatedData<Given = any>(data: Given, context: TActionContext): Promise<Given> {
       const entity = this.entity;
       if (!context) {
          throw new Error("Context must be provided for validation");
       }
 
-      const keys = Object.keys(data);
+      const keys = Object.keys(data as any);
       const validatedData: EntityData = {};
 
       // get relational references/keys
@@ -95,7 +101,7 @@ export class Mutator<DB> implements EmitsEvents {
          throw new Error(`No data left to update "${entity.name}"`);
       }
 
-      return validatedData;
+      return validatedData as Given;
    }
 
    protected async many(qb: MutatorQB): Promise<MutatorResponse> {
@@ -120,20 +126,23 @@ export class Mutator<DB> implements EmitsEvents {
       return { ...response, data: data[0]! };
    }
 
-   async insertOne(data: EntityData): Promise<MutatorResponse<EntityData>> {
+   async insertOne(data: Input): Promise<MutatorResponse<Output>> {
       const entity = this.entity;
       if (entity.type === "system" && this.__unstable_disable_system_entity_creation) {
          throw new Error(`Creation of system entity "${entity.name}" is disabled`);
       }
 
-      // @todo: establish the original order from "data"
+      const result = await this.emgr.emit(
+         new Mutator.Events.MutatorInsertBefore({ entity, data: data as any })
+      );
+
+      // if listener returned, take what's returned
+      const _data = result.returned ? result.params.data : data;
       const validatedData = {
          ...entity.getDefaultObject(),
-         ...(await this.getValidatedData(data, "create"))
+         ...(await this.getValidatedData(_data, "create"))
       };
 
-      await this.emgr.emit(new Mutator.Events.MutatorInsertBefore({ entity, data: validatedData }));
-
       // check if required fields are present
       const required = entity.getRequiredFields();
       for (const field of required) {
@@ -154,24 +163,29 @@ export class Mutator<DB> implements EmitsEvents {
 
       await this.emgr.emit(new Mutator.Events.MutatorInsertAfter({ entity, data: res.data }));
 
-      return res;
+      return res as any;
    }
 
-   async updateOne(id: PrimaryFieldType, data: EntityData): Promise<MutatorResponse<EntityData>> {
+   async updateOne(id: PrimaryFieldType, data: Partial<Input>): Promise<MutatorResponse<Output>> {
       const entity = this.entity;
       if (!Number.isInteger(id)) {
          throw new Error("ID must be provided for update");
       }
 
-      const validatedData = await this.getValidatedData(data, "update");
+      const result = await this.emgr.emit(
-
+         new Mutator.Events.MutatorUpdateBefore({
-      await this.emgr.emit(
+            entity,
-         new Mutator.Events.MutatorUpdateBefore({ entity, entityId: id, data: validatedData })
+            entityId: id,
+            data
+         })
       );
 
+      const _data = result.returned ? result.params.data : data;
+      const validatedData = await this.getValidatedData(_data, "update");
+
       const query = this.conn
          .updateTable(entity.name)
-         .set(validatedData)
+         .set(validatedData as any)
          .where(entity.id().name, "=", id)
          .returning(entity.getSelect());
 
@@ -181,10 +195,10 @@ export class Mutator<DB> implements EmitsEvents {
          new Mutator.Events.MutatorUpdateAfter({ entity, entityId: id, data: res.data })
       );
 
-      return res;
+      return res as any;
    }
 
-   async deleteOne(id: PrimaryFieldType): Promise<MutatorResponse<EntityData>> {
+   async deleteOne(id: PrimaryFieldType): Promise<MutatorResponse<Output>> {
       const entity = this.entity;
       if (!Number.isInteger(id)) {
          throw new Error("ID must be provided for deletion");
@@ -203,7 +217,7 @@ export class Mutator<DB> implements EmitsEvents {
          new Mutator.Events.MutatorDeleteAfter({ entity, entityId: id, data: res.data })
       );
 
-      return res;
+      return res as any;
    }
 
    private getValidOptions(options?: Partial<RepoQuery>): Partial<RepoQuery> {
@@ -250,47 +264,62 @@ export class Mutator<DB> implements EmitsEvents {
    }
 
    // @todo: decide whether entries should be deleted all at once or one by one (for events)
-   async deleteWhere(where?: RepoQuery["where"]): Promise<MutatorResponse<EntityData>> {
+   async deleteWhere(where?: RepoQuery["where"]): Promise<MutatorResponse<Output[]>> {
       const entity = this.entity;
 
       const qb = this.appendWhere(this.conn.deleteFrom(entity.name), where).returning(
          entity.getSelect()
       );
 
-      //await this.emgr.emit(new Mutator.Events.MutatorDeleteBefore({ entity, entityId: id }));
+      return (await this.many(qb)) as any;
-
-      const res = await this.many(qb);
-
-      /*await this.emgr.emit(
-         new Mutator.Events.MutatorDeleteAfter({ entity, entityId: id, data: res.data })
-      );*/
-
-      return res;
    }
 
    async updateWhere(
-      data: EntityData,
+      data: Partial<Input>,
       where?: RepoQuery["where"]
-   ): Promise<MutatorResponse<EntityData>> {
+   ): Promise<MutatorResponse<Output[]>> {
       const entity = this.entity;
-
       const validatedData = await this.getValidatedData(data, "update");
 
-      /*await this.emgr.emit(
-         new Mutator.Events.MutatorUpdateBefore({ entity, entityId: id, data: validatedData })
-      );*/
-
       const query = this.appendWhere(this.conn.updateTable(entity.name), where)
-         .set(validatedData)
+         .set(validatedData as any)
-         //.where(entity.id().name, "=", id)
          .returning(entity.getSelect());
 
-      const res = await this.many(query);
+      return (await this.many(query)) as any;
+   }
 
-      /*await this.emgr.emit(
+   async insertMany(data: Input[]): Promise<MutatorResponse<Output[]>> {
-         new Mutator.Events.MutatorUpdateAfter({ entity, entityId: id, data: res.data })
+      const entity = this.entity;
-      );*/
+      if (entity.type === "system" && this.__unstable_disable_system_entity_creation) {
+         throw new Error(`Creation of system entity "${entity.name}" is disabled`);
+      }
 
-      return res;
+      const validated: any[] = [];
+      for (const row of data) {
+         const validatedData = {
+            ...entity.getDefaultObject(),
+            ...(await this.getValidatedData(row, "create"))
+         };
+
+         // check if required fields are present
+         const required = entity.getRequiredFields();
+         for (const field of required) {
+            if (
+               typeof validatedData[field.name] === "undefined" ||
+               validatedData[field.name] === null
+            ) {
+               throw new Error(`Field "${field.name}" is required`);
+            }
+         }
+
+         validated.push(validatedData);
+      }
+
+      const query = this.conn
+         .insertInto(entity.name)
+         .values(validated)
+         .returning(entity.getSelect());
+
+      return (await this.many(query)) as any;
    }
 }

app/src/data/entities/query/Repository.ts

@@ -1,4 +1,4 @@
-import type { PrimaryFieldType } from "core";
+import type { DB as DefaultDB, PrimaryFieldType } from "core";
 import { type EmitsEvents, EventManager } from "core/events";
 import { type SelectQueryBuilder, sql } from "kysely";
 import { cloneDeep } from "lodash-es";
@@ -43,27 +43,29 @@ export type RepositoryExistsResponse = RepositoryRawResponse & {
    exists: boolean;
 };
 
-export class Repository<DB = any, TB extends keyof DB = any> implements EmitsEvents {
+export class Repository<TBD extends object = DefaultDB, TB extends keyof TBD = any>
-   em: EntityManager<DB>;
+   implements EmitsEvents
+{
+   em: EntityManager<TBD>;
    entity: Entity;
    static readonly Events = RepositoryEvents;
    emgr: EventManager<typeof Repository.Events>;
 
-   constructor(em: EntityManager<DB>, entity: Entity, emgr?: EventManager<any>) {
+   constructor(em: EntityManager<TBD>, entity: Entity, emgr?: EventManager<any>) {
       this.em = em;
       this.entity = entity;
       this.emgr = emgr ?? new EventManager(MutatorEvents);
    }
 
    private cloneFor(entity: Entity) {
-      return new Repository(this.em, entity, this.emgr);
+      return new Repository(this.em, this.em.entity(entity), this.emgr);
    }
 
    private get conn() {
       return this.em.connection.kysely;
    }
 
-   private getValidOptions(options?: Partial<RepoQuery>): RepoQuery {
+   getValidOptions(options?: Partial<RepoQuery>): RepoQuery {
       const entity = this.entity;
       // @todo: if not cloned deep, it will keep references and error if multiple requests come in
       const validated = {
@@ -92,23 +94,19 @@ export class Repository<DB = any, TB extends keyof DB = any> implements EmitsEve
          if (invalid.length > 0) {
             throw new InvalidSearchParamsException(
                `Invalid select field(s): ${invalid.join(", ")}`
-            );
+            ).context({
+               entity: entity.name,
+               valid: validated.select
+            });
          }
 
          validated.select = options.select;
       }
 
-      if (options.with && options.with.length > 0) {
+      if (options.with) {
-         for (const entry of options.with) {
+         const depth = WithBuilder.validateWiths(this.em, entity.name, options.with);
-            const related = this.em.relationOf(entity.name, entry);
+         // @todo: determine allowed depth
-            if (!related) {
+         validated.with = options.with;
-               throw new InvalidSearchParamsException(
-                  `WITH: "${entry}" is not a relation of "${entity.name}"`
-               );
-            }
-
-            validated.with.push(entry);
-         }
       }
 
       if (options.join && options.join.length > 0) {
@@ -230,49 +228,85 @@ export class Repository<DB = any, TB extends keyof DB = any> implements EmitsEve
       return { ...response, data: data[0]! };
    }
 
-   private buildQuery(
+   addOptionsToQueryBuilder(
+      _qb?: RepositoryQB,
       _options?: Partial<RepoQuery>,
-      exclude_options: (keyof RepoQuery)[] = []
+      config?: {
-   ): { qb: RepositoryQB; options: RepoQuery } {
+         validate?: boolean;
+         ignore?: (keyof RepoQuery)[];
+         alias?: string;
+         defaults?: Pick<RepoQuery, "limit" | "offset">;
+      }
+   ) {
       const entity = this.entity;
-      const options = this.getValidOptions(_options);
+      let qb = _qb ?? (this.conn.selectFrom(entity.name) as RepositoryQB);
 
-      const alias = entity.name;
+      const options = config?.validate !== false ? this.getValidOptions(_options) : _options;
+      if (!options) return qb;
+
+      const alias = config?.alias ?? entity.name;
       const aliased = (field: string) => `${alias}.${field}`;
-      let qb = this.conn
+      const ignore = config?.ignore ?? [];
-         .selectFrom(entity.name)
+      const defaults = {
-         .select(entity.getAliasedSelectFrom(options.select, alias));
+         limit: 10,
+         offset: 0,
+         ...config?.defaults
+      };
 
-      //console.log("build query options", options);
+      /*console.log("build query options", {
-      if (!exclude_options.includes("with") && options.with) {
+         entity: entity.name,
+         options,
+         config
+      });*/
+
+      if (!ignore.includes("select") && options.select) {
+         qb = qb.select(entity.getAliasedSelectFrom(options.select, alias));
+      }
+
+      if (!ignore.includes("with") && options.with) {
          qb = WithBuilder.addClause(this.em, qb, entity, options.with);
       }
 
-      if (!exclude_options.includes("join") && options.join) {
+      if (!ignore.includes("join") && options.join) {
          qb = JoinBuilder.addClause(this.em, qb, entity, options.join);
       }
 
       // add where if present
-      if (!exclude_options.includes("where") && options.where) {
+      if (!ignore.includes("where") && options.where) {
          qb = WhereBuilder.addClause(qb, options.where);
       }
 
-      if (!exclude_options.includes("limit")) qb = qb.limit(options.limit);
+      if (!ignore.includes("limit")) qb = qb.limit(options.limit ?? defaults.limit);
-      if (!exclude_options.includes("offset")) qb = qb.offset(options.offset);
+      if (!ignore.includes("offset")) qb = qb.offset(options.offset ?? defaults.offset);
 
       // sorting
-      if (!exclude_options.includes("sort")) {
+      if (!ignore.includes("sort")) {
-         qb = qb.orderBy(aliased(options.sort.by), options.sort.dir);
+         qb = qb.orderBy(aliased(options.sort?.by ?? "id"), options.sort?.dir ?? "asc");
       }
 
-      //console.log("options", { _options, options, exclude_options });
+      return qb as RepositoryQB;
-      return { qb, options };
+   }
+
+   private buildQuery(
+      _options?: Partial<RepoQuery>,
+      ignore: (keyof RepoQuery)[] = []
+   ): { qb: RepositoryQB; options: RepoQuery } {
+      const entity = this.entity;
+      const options = this.getValidOptions(_options);
+
+      return {
+         qb: this.addOptionsToQueryBuilder(undefined, options, {
+            ignore,
+            alias: entity.name
+         }),
+         options
+      };
    }
 
    async findId(
       id: PrimaryFieldType,
       _options?: Partial<Omit<RepoQuery, "where" | "limit" | "offset">>
-   ): Promise<RepositoryResponse<DB[TB]>> {
+   ): Promise<RepositoryResponse<TBD[TB] | undefined>> {
       const { qb, options } = this.buildQuery(
          {
             ..._options,
@@ -288,7 +322,7 @@ export class Repository<DB = any, TB extends keyof DB = any> implements EmitsEve
    async findOne(
       where: RepoQuery["where"],
       _options?: Partial<Omit<RepoQuery, "where" | "limit" | "offset">>
-   ): Promise<RepositoryResponse<DB[TB] | undefined>> {
+   ): Promise<RepositoryResponse<TBD[TB] | undefined>> {
       const { qb, options } = this.buildQuery({
          ..._options,
          where,
@@ -298,7 +332,7 @@ export class Repository<DB = any, TB extends keyof DB = any> implements EmitsEve
       return this.single(qb, options) as any;
    }
 
-   async findMany(_options?: Partial<RepoQuery>): Promise<RepositoryResponse<DB[TB][]>> {
+   async findMany(_options?: Partial<RepoQuery>): Promise<RepositoryResponse<TBD[TB][]>> {
       const { qb, options } = this.buildQuery(_options);
       //console.log("findMany:options", options);
 

app/src/data/entities/query/WhereBuilder.ts

@@ -30,7 +30,7 @@ function key(e: unknown): string {
    return e as string;
 }
 
-const expressions: TExpression<any, any, any>[] = [
+const expressions = [
    exp(
       "$eq",
       (v: Primitive) => isPrimitive(v),

app/src/data/entities/query/WithBuilder.ts

@@ -1,42 +1,82 @@
+import { isObject } from "core/utils";
+import type { KyselyJsonFrom, RepoQuery } from "data";
+import { InvalidSearchParamsException } from "data/errors";
 import type { Entity, EntityManager, RepositoryQB } from "../../entities";
 
 export class WithBuilder {
-   private static buildClause(
+   static addClause(
       em: EntityManager<any>,
       qb: RepositoryQB,
       entity: Entity,
-      withString: string
+      withs: RepoQuery["with"]
    ) {
-      const relation = em.relationOf(entity.name, withString);
+      if (!withs || !isObject(withs)) {
-      if (!relation) {
+         console.warn(`'withs' undefined or invalid, given: ${JSON.stringify(withs)}`);
-         throw new Error(`Relation "${withString}" not found`);
+         return qb;
       }
 
-      const cardinality = relation.ref(withString).cardinality;
-      //console.log("with--builder", { entity: entity.name, withString, cardinality });
-
       const fns = em.connection.fn;
-      const jsonFrom = cardinality === 1 ? fns.jsonObjectFrom : fns.jsonArrayFrom;
-
-      if (!jsonFrom) {
-         throw new Error("Connection does not support jsonObjectFrom/jsonArrayFrom");
-      }
-
-      try {
-         return relation.buildWith(entity, qb, jsonFrom, withString);
-      } catch (e) {
-         throw new Error(`Could not build "with" relation "${withString}": ${(e as any).message}`);
-      }
-   }
-
-   static addClause(em: EntityManager<any>, qb: RepositoryQB, entity: Entity, withs: string[]) {
-      if (withs.length === 0) return qb;
-
       let newQb = qb;
-      for (const entry of withs) {
+
-         newQb = WithBuilder.buildClause(em, newQb, entity, entry);
+      for (const [ref, query] of Object.entries(withs)) {
+         const relation = em.relationOf(entity.name, ref);
+         if (!relation) {
+            throw new Error(`Relation "${entity.name}<>${ref}" not found`);
+         }
+         const cardinality = relation.ref(ref).cardinality;
+         const jsonFrom: KyselyJsonFrom =
+            cardinality === 1 ? fns.jsonObjectFrom : fns.jsonArrayFrom;
+         if (!jsonFrom) {
+            throw new Error("Connection does not support jsonObjectFrom/jsonArrayFrom");
+         }
+
+         const other = relation.other(entity);
+         newQb = newQb.select((eb) => {
+            let subQuery = relation.buildWith(entity, ref)(eb);
+            if (query) {
+               subQuery = em.repo(other.entity).addOptionsToQueryBuilder(subQuery, query as any, {
+                  ignore: ["with", "join", cardinality === 1 ? "limit" : undefined].filter(
+                     Boolean
+                  ) as any
+               });
+            }
+
+            if (query.with) {
+               subQuery = WithBuilder.addClause(em, subQuery, other.entity, query.with as any);
+            }
+
+            return jsonFrom(subQuery).as(other.reference);
+         });
       }
 
       return newQb;
    }
+
+   static validateWiths(em: EntityManager<any>, entity: string, withs: RepoQuery["with"]) {
+      let depth = 0;
+      if (!withs || !isObject(withs)) {
+         withs && console.warn(`'withs' invalid, given: ${JSON.stringify(withs)}`);
+         return depth;
+      }
+
+      const child_depths: number[] = [];
+      for (const [ref, query] of Object.entries(withs)) {
+         const related = em.relationOf(entity, ref);
+         if (!related) {
+            throw new InvalidSearchParamsException(
+               `WITH: "${ref}" is not a relation of "${entity}"`
+            );
+         }
+         depth++;
+
+         if ("with" in query) {
+            child_depths.push(WithBuilder.validateWiths(em, ref, query.with as any));
+         }
+      }
+      if (child_depths.length > 0) {
+         depth += Math.max(...child_depths);
+      }
+
+      return depth;
+   }
 }

app/src/data/events/index.ts

@@ -1,20 +1,48 @@
 import type { PrimaryFieldType } from "core";
-import { Event } from "core/events";
+import { Event, InvalidEventReturn } from "core/events";
 import type { Entity, EntityData } from "../entities";
 import type { RepoQuery } from "../server/data-query-impl";
 
-export class MutatorInsertBefore extends Event<{ entity: Entity; data: EntityData }> {
+export class MutatorInsertBefore extends Event<{ entity: Entity; data: EntityData }, EntityData> {
    static override slug = "mutator-insert-before";
+
+   override validate(data: EntityData) {
+      const { entity } = this.params;
+      if (!entity.isValidData(data, "create")) {
+         throw new InvalidEventReturn("EntityData", "invalid");
+      }
+
+      return this.clone({
+         entity,
+         data
+      });
+   }
 }
 export class MutatorInsertAfter extends Event<{ entity: Entity; data: EntityData }> {
    static override slug = "mutator-insert-after";
 }
-export class MutatorUpdateBefore extends Event<{
+export class MutatorUpdateBefore extends Event<
-   entity: Entity;
+   {
-   entityId: PrimaryFieldType;
+      entity: Entity;
-   data: EntityData;
+      entityId: PrimaryFieldType;
-}> {
+      data: EntityData;
+   },
+   EntityData
+> {
    static override slug = "mutator-update-before";
+
+   override validate(data: EntityData) {
+      const { entity, ...rest } = this.params;
+      if (!entity.isValidData(data, "update")) {
+         throw new InvalidEventReturn("EntityData", "invalid");
+      }
+
+      return this.clone({
+         ...rest,
+         entity,
+         data
+      });
+   }
 }
 export class MutatorUpdateAfter extends Event<{
    entity: Entity;