Merge pull request #58 from bknd-io/release/0.6.2

Release 0.6.2

.gitignore

@@ -18,6 +18,7 @@ packages/media/.env
 **/*/vite.config.ts.timestamp*
 .history
 **/*/.db/*
+**/*/.configs/*
 **/*/*.db
 **/*/*.db-shm
 **/*/*.db-wal

LICENSE.md

@@ -6,7 +6,7 @@ FSL-1.1-MIT
 
 ## Notice
 
-Copyright 2024 Webintex GmbH
+Copyright 2025 Dennis Senn
 
 ## Terms and Conditions
 

README.md

@@ -1,9 +1,15 @@
-![bknd](https://raw.githubusercontent.com/bknd-io/bknd/refs/heads/main/app/src/ui/assets/poster.png)
+[![npm version](https://img.shields.io/npm/v/bknd.svg)](https://npmjs.org/package/bknd)
+[![npm downloads](https://img.shields.io/npm/dm/bknd)](https://www.npmjs.com/package/bknd)
 
-bknd simplifies app development by providing fully functional backend for data management, 
-authentication, workflows and media. Since it's lightweight and built on Web Standards, it can 
-be deployed nearly anywhere, including running inside your framework of choice. No more 
-deploying multiple separate services!
+![bknd](https://raw.githubusercontent.com/bknd-io/bknd/refs/heads/main/docs/_assets/poster.png)
+
+<p align="center" width="100%">
+<a href="https://stackblitz.com/github/bknd-io/bknd-examples?hideExplorer=1&embed=1&view=preview&startScript=example-admin-rich&initialPath=%2Fdata%2Fschema">
+<strong>⭐ Live Demo</strong>
+</a>
+</p>
+
+bknd simplifies app development by providing a fully functional backend for database management, authentication, media and workflows. Being lightweight and built on Web Standards, it can be deployed nearly anywhere, including running inside your framework of choice. No more deploying multiple separate services!
 
 **For documentation and examples, please visit https://docs.bknd.io.**
 
@@ -11,33 +17,116 @@ deploying multiple separate services!
 > Please keep in mind that **bknd** is still under active development
 > and therefore full backward compatibility is not guaranteed before reaching v1.0.0.
 
-## ✨ Features
-**📊 Data**: Define, query, and control your data with ease. 
-  - Define entities with fields and relationships, synced directly to your database.  
-  - Supported field types: `primary`, `text`, `number`, `date`, `boolean`, `enum`, `json`, `jsonschema`.  
-  - Relationship types: `one-to-one`, `many-to-one`, `many-to-many`, and `polymorphic`.  
-  - Advanced querying with the **Repository**: filtering, sorting, pagination, and relational data handling.
-  - Seamlessly manage data with mutators and a robust event system.  
-  - Extend database functionality with batching, introspection, and support for multiple SQL dialects.
+## Size
+![gzipped size of bknd](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/index.js?compression=gzip&label=bknd)
+![gzipped size of bknd/client](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/ui/client/index.js?compression=gzip&label=bknd/client)
+![gzipped size of bknd/elements](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/ui/elements/index.js?compression=gzip&label=bknd/elements)
+![gzipped size of bknd/ui](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/ui/index.js?compression=gzip&label=bknd/ui)
 
-**🔐 Auth**: Easily implement reliable authentication strategies.
-  - Built-in `user` entity with customizable fields.  
-  - Supports multiple authentication strategies:  
-    - Email/password (with hashed storage).  
-    - OAuth/OIDC (Google, GitHub, and more).  
-  - Secure JWT generation and session management.
+The size on npm is misleading, as the `bknd` package includes the backend, the ui components as well as the whole backend bundled into the cli including static assets. 
 
-**🖼️ Media**: Effortlessly manage and serve all your media files.
-  - Upload files with ease.  
-  - Adapter-based support for S3, S3-compatible storage (e.g., R2, Tigris), and Cloudinary.
+## Motivation
+Creating digital products always requires developing both the backend (the logic) and the frontend (the appearance). Building a backend from scratch demands deep knowledge in areas such as authentication and database management. Using a backend framework can speed up initial development, but it still requires ongoing effort to work within its constraints (e.g., *"how to do X with Y?"*), which can quickly slow you down. Choosing a backend system is a tough decision, as you might not be aware of its limitations until you encounter them.
 
-**🔄 Flows** (UI integration coming soon!): Design and run workflows with seamless automation.
-  - Create and run workflows with trigger-based automation:  
-    - Manual triggers or events from data, auth, media, or server actions.  
-    - HTTP triggers for external integrations.  
-  - Define tasks in sequence, parallel, or loops, with conditional execution.  
-  - Use reusable sub-workflows to organize complex processes.  
-  - Leverage OpenAPI specifications for API-based tasks.
+**The solution:** A backend system that only assumes and implements primitive details, integrates into multiple environments, and adheres to industry standards.
+
+## Features
+* ⚡ Instant backend with full REST API:
+  * **Data**: Define, query, and control your data with ease.
+  * **Auth**: Easily implement reliable authentication strategies.
+  * **Media**: Effortlessly manage and serve all your media files.
+  * **Flows**: Design and run workflows with seamless automation. (UI integration coming soon!)
+* 🌐 Built on Web Standards for maximum compatibility
+* 🏃‍♂️ Multiple run modes
+  * standalone using the CLI
+  * using a JavaScript runtime (Node, Bun, workerd)
+  * using a React framework (Astro, Remix, Next.js)
+* 📦 Official API and React SDK with type-safety
+* ⚛️ React elements for auto-configured authentication and media components
+
+## Structure
+The package is mainly split into 4 parts, each serving a specific purpose:
+
+| Import                      | Purpose                                              |
+|-----------------------------|------------------------------------------------------|
+| `bknd`<br/>`bknd/adapter/*` | Backend including APIs and adapters                  |
+| `bknd/ui`                   | Admin UI components for react frameworks             |
+| `bknd/client`               | TypeScript SDK and React hooks for the API endpoints |
+| `bknd/elements`             | React components for authentication and media        |
+
+
+### The backend (`bknd`)
+Serve the backend as an API for any JS runtime or framework. The latter is especially handy, as it allows you to deploy your frontend and backend bundled together. Furthermore it allows adding additional logic in a way you're already familar with. Just add another route and you're good to go.
+
+Here is an example of serving the API using node:
+```js index.js
+import { serve } from "bknd/adapter/node"
+serve();
+```
+
+### Integrated admin UI (`bknd/ui`)
+The admin UI allows to manage your data including full configuration of your backend using a graphical user interface. Using `vite`, your admin route looks like this:
+```tsx
+import { Admin } from "bknd/ui"
+import "bknd/dist/styles.css";
+
+export default function AdminPage() {
+   return <Admin />
+}
+```
+
+### Using the REST API or TypeScript SDK (`bknd/client`)
+If you're not using a JavaScript environment, you can still access any endpoint using the REST API:
+```bash
+curl -XGET <your-endpoint>/api/data/<entity>
+{
+  "data": [
+    { "id": 1, ... },
+    { "id": 2, ... }
+  ],
+  "meta": { /* ... */ }
+}
+```
+
+In a JavaScript environment, you can use the TypeScript SDK with type-safety. The above example would look like this:
+```ts
+import { Api } from "bknd/client";
+
+const api = new Api({ host: "<endpoint>" });
+const { data } = await api.data.readMany("<entity>");
+```
+
+If you're using React, there are 2 hooks exposed (`useApi`, `useEntity`), as well as an `swr` wrapper around each (`useApiQuery`, `useEntityQuery`). The `swr` wrapped hooks automatically handled query invalidation:
+
+```tsx
+import { useState } from "react";
+import { useEntityQuery } from "bknd/client";
+
+export default function App() {
+   const { data } = useEntityQuery("todos");   
+   return <ul>
+      {data?.map(todo => (
+         <li key={todo.id}>{todo.name}</li>
+      ))}
+   </ul>
+}
+```
+
+### React elements (`bknd/elements`)
+You don't have to figure out API details to include media uploads to your app. For an user avatar upload, this is all you need:
+```tsx
+import { Media } from "bknd/elements"
+import "bknd/dist/main.css"
+
+export function UserAvatar() {
+   return <Media.Dropzone
+     entity={{ name: "users", id: 1, field: "avatar" }}
+     maxItems={1}
+     overwrite
+   />
+}
+```
+The import path also exports components for login and registration forms which are automatically pointed to the `bknd` defaults.
 
 
 ## 🚀 Quick start

app/__test__/Module.spec.ts

@@ -1,38 +0,0 @@
-import { describe, expect, test } from "bun:test";
-import { type TSchema, Type, stripMark } from "../src/core/utils";
-import { Module } from "../src/modules/Module";
-
-function createModule<Schema extends TSchema>(schema: Schema) {
-   class TestModule extends Module<typeof schema> {
-      getSchema() {
-         return schema;
-      }
-      toJSON() {
-         return this.config;
-      }
-      useForceParse() {
-         return true;
-      }
-   }
-
-   return TestModule;
-}
-
-describe("Module", async () => {
-   test("basic", async () => {});
-
-   test("listener", async () => {
-      let result: any;
-
-      const module = createModule(Type.Object({ a: Type.String() }));
-      const m = new module({ a: "test" });
-
-      await m.schema().set({ a: "test2" });
-      m.setListener(async (c) => {
-         await new Promise((r) => setTimeout(r, 10));
-         result = stripMark(c);
-      });
-      await m.schema().set({ a: "test3" });
-      expect(result).toEqual({ a: "test3" });
-   });
-});

app/__test__/api/DataApi.spec.ts

@@ -0,0 +1,70 @@
+import { afterAll, beforeAll, describe, expect, it } from "bun:test";
+import { Guard } from "../../src/auth";
+import { parse } from "../../src/core/utils";
+import { DataApi } from "../../src/data/api/DataApi";
+import { DataController } from "../../src/data/api/DataController";
+import { dataConfigSchema } from "../../src/data/data-schema";
+import * as proto from "../../src/data/prototype";
+import { disableConsoleLog, enableConsoleLog, schemaToEm } from "../helper";
+
+beforeAll(disableConsoleLog);
+afterAll(enableConsoleLog);
+
+const dataConfig = parse(dataConfigSchema, {});
+describe("DataApi", () => {
+   it("should switch to post for long url reads", async () => {
+      const api = new DataApi();
+
+      const get = api.readMany("a".repeat(300), { select: ["id", "name"] });
+      expect(get.request.method).toBe("GET");
+      expect(new URL(get.request.url).pathname).toBe(`/api/data/${"a".repeat(300)}`);
+
+      const post = api.readMany("a".repeat(1000), { select: ["id", "name"] });
+      expect(post.request.method).toBe("POST");
+      expect(new URL(post.request.url).pathname).toBe(`/api/data/${"a".repeat(1000)}/query`);
+   });
+
+   it("returns result", async () => {
+      const schema = proto.em({
+         posts: proto.entity("posts", { title: proto.text() })
+      });
+      const em = schemaToEm(schema);
+      await em.schema().sync({ force: true });
+
+      const payload = [{ title: "foo" }, { title: "bar" }, { title: "baz" }];
+      await em.mutator("posts").insertMany(payload);
+
+      const ctx: any = { em, guard: new Guard() };
+      const controller = new DataController(ctx, dataConfig);
+      const app = controller.getController();
+
+      {
+         const res = (await app.request("/posts")) as Response;
+         const { data } = await res.json();
+         expect(data.length).toEqual(3);
+      }
+
+      // @ts-ignore tests
+      const api = new DataApi({ basepath: "/", queryLengthLimit: 50 });
+      // @ts-ignore protected
+      api.fetcher = app.request as typeof fetch;
+      {
+         const req = api.readMany("posts", { select: ["title"] });
+         expect(req.request.method).toBe("GET");
+         const res = await req;
+         expect(res.data).toEqual(payload);
+      }
+
+      {
+         const req = api.readMany("posts", {
+            select: ["title"],
+            limit: 100000,
+            offset: 0,
+            sort: "id"
+         });
+         expect(req.request.method).toBe("POST");
+         const res = await req;
+         expect(res.data).toEqual(payload);
+      }
+   });
+});

app/__test__/api/ModuleApi.spec.ts

@@ -28,6 +28,8 @@ describe("ModuleApi", () => {
    it("fetches endpoint", async () => {
       const app = new Hono().get("/endpoint", (c) => c.json({ foo: "bar" }));
       const api = new Api({ host });
+
+      // @ts-expect-error it's protected
       api.fetcher = app.request as typeof fetch;
 
       const res = await api.get("/endpoint");
@@ -40,6 +42,8 @@ describe("ModuleApi", () => {
    it("has accessible request", async () => {
       const app = new Hono().get("/endpoint", (c) => c.json({ foo: "bar" }));
       const api = new Api({ host });
+
+      // @ts-expect-error it's protected
       api.fetcher = app.request as typeof fetch;
 
       const promise = api.get("/endpoint");

app/__test__/app/repro.spec.ts

@@ -0,0 +1,73 @@
+import { describe, expect, test } from "bun:test";
+import { createApp, registries } from "../../src";
+import * as proto from "../../src/data/prototype";
+import { StorageLocalAdapter } from "../../src/media/storage/adapters/StorageLocalAdapter";
+
+describe("repros", async () => {
+   /**
+    * steps:
+    * 1. enable media
+    * 2. create 'test' entity
+    * 3. add media to 'test'
+    *
+    * There was an issue that AppData had old configs because of system entity "media"
+    */
+   test("registers media entity correctly to relate to it", async () => {
+      registries.media.register("local", StorageLocalAdapter);
+      const app = createApp();
+      await app.build();
+
+      {
+         // 1. enable media
+         const [, config] = await app.module.media.schema().patch("", {
+            enabled: true,
+            adapter: {
+               type: "local",
+               config: {
+                  path: "./"
+               }
+            }
+         });
+
+         expect(config.enabled).toBe(true);
+      }
+
+      {
+         // 2. create 'test' entity
+         await app.module.data.schema().patch(
+            "entities.test",
+            proto
+               .entity("test", {
+                  content: proto.text()
+               })
+               .toJSON()
+         );
+         expect(app.em.entities.map((e) => e.name)).toContain("test");
+      }
+
+      {
+         await app.module.data.schema().patch("entities.test.fields.files", {
+            type: "media",
+            config: {
+               required: false,
+               fillable: ["update"],
+               hidden: false,
+               mime_types: [],
+               virtual: true,
+               entity: "test"
+            }
+         });
+
+         expect(
+            app.module.data.schema().patch("relations.000", {
+               type: "poly",
+               source: "test",
+               target: "media",
+               config: { mappedBy: "files" }
+            })
+         ).resolves.toBeDefined();
+      }
+
+      expect(app.em.entities.map((e) => e.name)).toEqual(["media", "test"]);
+   });
+});

app/__test__/auth/middleware.spec.ts

@@ -0,0 +1 @@
+import { describe, expect, it } from "bun:test";

app/__test__/core/EventManager.spec.ts

@@ -1,8 +1,12 @@
-import { describe, expect, test } from "bun:test";
-import { Event, EventManager, NoParamEvent } from "../../src/core/events";
+import { afterAll, beforeAll, describe, expect, mock, test } from "bun:test";
+import { Event, EventManager, InvalidEventReturn, NoParamEvent } from "../../src/core/events";
+import { disableConsoleLog, enableConsoleLog } from "../helper";
+
+beforeAll(disableConsoleLog);
+afterAll(enableConsoleLog);
 
 class SpecialEvent extends Event<{ foo: string }> {
-   static slug = "special-event";
+   static override slug = "special-event";
 
    isBar() {
       return this.params.foo === "bar";
@@ -10,37 +14,139 @@ class SpecialEvent extends Event<{ foo: string }> {
 }
 
 class InformationalEvent extends NoParamEvent {
-   static slug = "informational-event";
+   static override slug = "informational-event";
+}
+
+class ReturnEvent extends Event<{ foo: string }, string> {
+   static override slug = "return-event";
+
+   override validate(value: string) {
+      if (typeof value !== "string") {
+         throw new InvalidEventReturn("string", typeof value);
+      }
+
+      return this.clone({
+         foo: [this.params.foo, value].join("-")
+      });
+   }
 }
 
 describe("EventManager", async () => {
-   test("test", async () => {
+   test("executes", async () => {
+      const call = mock(() => null);
+      const delayed = mock(() => null);
+
       const emgr = new EventManager();
       emgr.registerEvents([SpecialEvent, InformationalEvent]);
 
+      expect(emgr.eventExists("special-event")).toBe(true);
+      expect(emgr.eventExists("informational-event")).toBe(true);
+      expect(emgr.eventExists("unknown-event")).toBe(false);
+
       emgr.onEvent(
          SpecialEvent,
          async (event, name) => {
-            console.log("Event: ", name, event.params.foo, event.isBar());
-            console.log("wait...");
-
-            await new Promise((resolve) => setTimeout(resolve, 100));
-            console.log("done waiting");
+            expect(name).toBe("special-event");
+            expect(event.isBar()).toBe(true);
+            call();
+            await new Promise((resolve) => setTimeout(resolve, 50));
+            delayed();
          },
          "sync"
       );
 
+      // don't allow unknown
+      expect(() => emgr.on("unknown", () => void 0)).toThrow();
+
       emgr.onEvent(InformationalEvent, async (event, name) => {
-         console.log("Event: ", name, event.params);
+         call();
+         expect(name).toBe("informational-event");
       });
 
       await emgr.emit(new SpecialEvent({ foo: "bar" }));
-      console.log("done");
+      await emgr.emit(new InformationalEvent());
 
       // expect construct signatures to not cause ts errors
       new SpecialEvent({ foo: "bar" });
       new InformationalEvent();
 
-      expect(true).toBe(true);
+      expect(call).toHaveBeenCalledTimes(2);
+      expect(delayed).toHaveBeenCalled();
+   });
+
+   test("custom async executor", async () => {
+      const call = mock(() => null);
+      const asyncExecutor = (p: Promise<any>[]) => {
+         call();
+         return Promise.all(p);
+      };
+      const emgr = new EventManager(
+         { InformationalEvent },
+         {
+            asyncExecutor
+         }
+      );
+
+      emgr.onEvent(InformationalEvent, async () => {});
+      await emgr.emit(new InformationalEvent());
+      expect(call).toHaveBeenCalled();
+   });
+
+   test("piping", async () => {
+      const onInvalidReturn = mock(() => null);
+      const asyncEventCallback = mock(() => null);
+      const emgr = new EventManager(
+         { ReturnEvent, InformationalEvent },
+         {
+            onInvalidReturn
+         }
+      );
+
+      // @ts-expect-error InformationalEvent has no return value
+      emgr.onEvent(InformationalEvent, async () => {
+         asyncEventCallback();
+         return 1;
+      });
+
+      emgr.onEvent(ReturnEvent, async () => "1", "sync");
+      emgr.onEvent(ReturnEvent, async () => "0", "sync");
+
+      // @ts-expect-error must be string
+      emgr.onEvent(ReturnEvent, async () => 0, "sync");
+
+      // return is not required
+      emgr.onEvent(ReturnEvent, async () => {}, "sync");
+
+      // was "async", will not return
+      const e1 = await emgr.emit(new InformationalEvent());
+      expect(e1.returned).toBe(false);
+
+      const e2 = await emgr.emit(new ReturnEvent({ foo: "bar" }));
+      expect(e2.returned).toBe(true);
+      expect(e2.params.foo).toBe("bar-1-0");
+      expect(onInvalidReturn).toHaveBeenCalled();
+      expect(asyncEventCallback).toHaveBeenCalled();
+   });
+
+   test("once", async () => {
+      const call = mock(() => null);
+      const emgr = new EventManager({ InformationalEvent });
+
+      emgr.onEvent(
+         InformationalEvent,
+         async (event, slug) => {
+            expect(event).toBeInstanceOf(InformationalEvent);
+            expect(slug).toBe("informational-event");
+            call();
+         },
+         { mode: "sync", once: true }
+      );
+
+      expect(emgr.getListeners().length).toBe(1);
+      await emgr.emit(new InformationalEvent());
+      expect(emgr.getListeners().length).toBe(0);
+      await emgr.emit(new InformationalEvent());
+      expect(emgr.getListeners().length).toBe(0);
+      expect(call).toHaveBeenCalledTimes(1);
    });
 });

app/__test__/core/crypto.spec.ts

@@ -1,14 +1,16 @@
-import { describe, test } from "bun:test";
+import { describe, expect, test } from "bun:test";
 import { checksum, hash } from "../../src/core/utils";
 
 describe("crypto", async () => {
    test("sha256", async () => {
-      console.log(await hash.sha256("test"));
+      expect(await hash.sha256("test")).toBe(
+         "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
+      );
    });
    test("sha1", async () => {
-      console.log(await hash.sha1("test"));
+      expect(await hash.sha1("test")).toBe("a94a8fe5ccb19ba61c4c0873d391e987982fbbd3");
    });
    test("checksum", async () => {
-      console.log(checksum("hello world"));
+      expect(await checksum("hello world")).toBe("2aae6c35c94fcfb415dbe95f408b9ce91ee846ed");
    });
 });

app/__test__/data/data-query-impl.spec.ts

@@ -1,16 +1,21 @@
 import { describe, expect, test } from "bun:test";
-import type { QueryObject } from "ufo";
-import { WhereBuilder, type WhereQuery } from "../../src/data/entities/query/WhereBuilder";
+import { Value, _jsonp } from "../../src/core/utils";
+import { type RepoQuery, WhereBuilder, type WhereQuery, querySchema } from "../../src/data";
+import type { RepoQueryIn } from "../../src/data/server/data-query-impl";
 import { getDummyConnection } from "./helper";
 
-const t = "t";
+const decode = (input: RepoQueryIn, expected: RepoQuery) => {
+   const result = Value.Decode(querySchema, input);
+   expect(result).toEqual(expected);
+};
+
 describe("data-query-impl", () => {
    function qb() {
       const c = getDummyConnection();
       const kysely = c.dummyConnection.kysely;
-      return kysely.selectFrom(t).selectAll();
+      return kysely.selectFrom("t").selectAll();
    }
-   function compile(q: QueryObject) {
+   function compile(q: WhereQuery) {
       const { sql, parameters } = WhereBuilder.addClause(qb(), q).compile();
       return { sql, parameters };
    }
@@ -89,4 +94,47 @@ describe("data-query-impl", () => {
          expect(keys).toEqual(expectedKeys);
       }
    });
+
+   test("with", () => {
+      decode({ with: ["posts"] }, { with: { posts: {} } });
+      decode({ with: { posts: {} } }, { with: { posts: {} } });
+      decode({ with: { posts: { limit: 1 } } }, { with: { posts: { limit: 1 } } });
+      decode(
+         {
+            with: {
+               posts: {
+                  with: {
+                     images: {
+                        select: ["id"]
+                     }
+                  }
+               }
+            }
+         },
+         {
+            with: {
+               posts: {
+                  with: {
+                     images: {
+                        select: ["id"]
+                     }
+                  }
+               }
+            }
+         }
+      );
+   });
+});
+
+describe("data-query-impl: Typebox", () => {
+   test("sort", async () => {
+      const _dflt = { sort: { by: "id", dir: "asc" } };
+
+      decode({ sort: "" }, _dflt);
+      decode({ sort: "name" }, { sort: { by: "name", dir: "asc" } });
+      decode({ sort: "-name" }, { sort: { by: "name", dir: "desc" } });
+      decode({ sort: "-posts.name" }, { sort: { by: "posts.name", dir: "desc" } });
+      decode({ sort: "-1name" }, _dflt);
+      decode({ sort: { by: "name", dir: "desc" } }, { sort: { by: "name", dir: "desc" } });
+   });
 });

app/__test__/data/data.test.ts

@@ -18,7 +18,7 @@ describe("some tests", async () => {
 
    const users = new Entity("users", [
       new TextField("username", { required: true, default_value: "nobody" }),
-      new TextField("email", { max_length: 3 })
+      new TextField("email", { maxLength: 3 })
    ]);
 
    const posts = new Entity("posts", [

app/__test__/data/relations.test.ts

@@ -106,7 +106,6 @@ describe("Relations", async () => {
       expect(postAuthorRel?.other(posts).entity).toBe(users);
 
       const kysely = em.connection.kysely;
-      const jsonFrom = (e) => e;
       /**
        * Relation Helper
        */
@@ -119,14 +118,11 @@ describe("Relations", async () => {
        - select: users.*
        - cardinality: 1
        */
-      const selectPostsFromUsers = postAuthorRel.buildWith(
-         users,
-         kysely.selectFrom(users.name),
-         jsonFrom,
-         "posts"
-      );
+      const selectPostsFromUsers = kysely
+         .selectFrom(users.name)
+         .select((eb) => postAuthorRel.buildWith(users, "posts")(eb).as("posts"));
       expect(selectPostsFromUsers.compile().sql).toBe(
-         'select (select "posts"."id" as "id", "posts"."title" as "title", "posts"."author_id" as "author_id" from "posts" as "posts" where "posts"."author_id" = "users"."id" limit ?) as "posts" from "users"'
+         'select (select from "posts" as "posts" where "posts"."author_id" = "users"."id") as "posts" from "users"'
       );
       expect(postAuthorRel!.getField()).toBeInstanceOf(RelationField);
       const userObj = { id: 1, username: "test" };
@@ -141,15 +137,12 @@ describe("Relations", async () => {
        - select: posts.*
        - cardinality:
        */
-      const selectUsersFromPosts = postAuthorRel.buildWith(
-         posts,
-         kysely.selectFrom(posts.name),
-         jsonFrom,
-         "author"
-      );
+      const selectUsersFromPosts = kysely
+         .selectFrom(posts.name)
+         .select((eb) => postAuthorRel.buildWith(posts, "author")(eb).as("author"));
 
       expect(selectUsersFromPosts.compile().sql).toBe(
-         'select (select "author"."id" as "id", "author"."username" as "username" from "users" as "author" where "author"."id" = "posts"."author_id" limit ?) as "author" from "posts"'
+         'select (select from "users" as "author" where "author"."id" = "posts"."author_id" limit ?) as "author" from "posts"'
       );
       expect(postAuthorRel.getField()).toBeInstanceOf(RelationField);
       const postObj = { id: 1, title: "test" };
@@ -315,20 +308,16 @@ describe("Relations", async () => {
        - select: users.*
        - cardinality: 1
        */
-      const selectCategoriesFromPosts = postCategoriesRel.buildWith(
-         posts,
-         kysely.selectFrom(posts.name),
-         jsonFrom
-      );
+      const selectCategoriesFromPosts = kysely
+         .selectFrom(posts.name)
+         .select((eb) => postCategoriesRel.buildWith(posts)(eb).as("categories"));
       expect(selectCategoriesFromPosts.compile().sql).toBe(
          'select (select "categories"."id" as "id", "categories"."label" as "label" from "categories" inner join "posts_categories" on "categories"."id" = "posts_categories"."categories_id" where "posts"."id" = "posts_categories"."posts_id" limit ?) as "categories" from "posts"'
       );
 
-      const selectPostsFromCategories = postCategoriesRel.buildWith(
-         categories,
-         kysely.selectFrom(categories.name),
-         jsonFrom
-      );
+      const selectPostsFromCategories = kysely
+         .selectFrom(categories.name)
+         .select((eb) => postCategoriesRel.buildWith(categories)(eb).as("posts"));
       expect(selectPostsFromCategories.compile().sql).toBe(
          'select (select "posts"."id" as "id", "posts"."title" as "title" from "posts" inner join "posts_categories" on "posts"."id" = "posts_categories"."posts_id" where "categories"."id" = "posts_categories"."categories_id" limit ?) as "posts" from "categories"'
       );

app/__test__/data/specs/Mutator.spec.ts

@@ -1,4 +1,5 @@
 import { afterAll, describe, expect, test } from "bun:test";
+import type { EventManager } from "../../../src/core/events";
 import {
    Entity,
    EntityManager,
@@ -10,6 +11,7 @@ import {
    RelationMutator,
    TextField
 } from "../../../src/data";
+import * as proto from "../../../src/data/prototype";
 import { getDummyConnection } from "../helper";
 
 const { dummyConnection, afterAllCleanup } = getDummyConnection();
@@ -83,14 +85,12 @@ describe("[data] Mutator (ManyToOne)", async () => {
 
       // persisting reference should ...
       expect(
-         postRelMutator.persistReference(relations[0], "users", {
+         postRelMutator.persistReference(relations[0]!, "users", {
             $set: { id: userData.data.id }
          })
       ).resolves.toEqual(["users_id", userData.data.id]);
       // @todo: add what methods are allowed to relation, like $create should not be allowed for post<>users
 
-      process.exit(0);
-
       const userRelMutator = new RelationMutator(users, em);
       expect(userRelMutator.getRelationalKeys()).toEqual(["posts"]);
    });
@@ -99,7 +99,7 @@ describe("[data] Mutator (ManyToOne)", async () => {
       expect(
          em.mutator(posts).insertOne({
             title: "post1",
-            users_id: 1 // user does not exist yet
+            users_id: 100 // user does not exist yet
          })
       ).rejects.toThrow();
    });
@@ -299,4 +299,71 @@ describe("[data] Mutator (Events)", async () => {
       expect(events.has(MutatorEvents.MutatorDeleteBefore.slug)).toBeTrue();
       expect(events.has(MutatorEvents.MutatorDeleteAfter.slug)).toBeTrue();
    });
+
+   test("insertOne event return is respected", async () => {
+      const posts = proto.entity("posts", {
+         title: proto.text(),
+         views: proto.number()
+      });
+
+      const conn = getDummyConnection();
+      const em = new EntityManager([posts], conn.dummyConnection);
+      await em.schema().sync({ force: true });
+
+      const emgr = em.emgr as EventManager<any>;
+
+      emgr.onEvent(
+         // @ts-ignore
+         EntityManager.Events.MutatorInsertBefore,
+         async (event) => {
+            return {
+               ...event.params.data,
+               views: 2
+            };
+         },
+         "sync"
+      );
+
+      const mutator = em.mutator("posts");
+      const result = await mutator.insertOne({ title: "test", views: 1 });
+      expect(result.data).toEqual({
+         id: 1,
+         title: "test",
+         views: 2
+      });
+   });
+
+   test("updateOne event return is respected", async () => {
+      const posts = proto.entity("posts", {
+         title: proto.text(),
+         views: proto.number()
+      });
+
+      const conn = getDummyConnection();
+      const em = new EntityManager([posts], conn.dummyConnection);
+      await em.schema().sync({ force: true });
+
+      const emgr = em.emgr as EventManager<any>;
+
+      emgr.onEvent(
+         // @ts-ignore
+         EntityManager.Events.MutatorUpdateBefore,
+         async (event) => {
+            return {
+               ...event.params.data,
+               views: event.params.data.views + 1
+            };
+         },
+         "sync"
+      );
+
+      const mutator = em.mutator("posts");
+      const created = await mutator.insertOne({ title: "test", views: 1 });
+      const result = await mutator.updateOne(created.data.id, { views: 2 });
+      expect(result.data).toEqual({
+         id: 1,
+         title: "test",
+         views: 3
+      });
+   });
 });

app/__test__/data/specs/Repository.spec.ts

@@ -1,7 +1,6 @@
 import { afterAll, describe, expect, test } from "bun:test";
-// @ts-ignore
-import { Perf } from "@bknd/core/utils";
 import type { Kysely, Transaction } from "kysely";
+import { Perf } from "../../../src/core/utils";
 import {
    Entity,
    EntityManager,
@@ -24,7 +23,7 @@ async function sleep(ms: number) {
 }
 
 describe("[Repository]", async () => {
-   test("bulk", async () => {
+   test.skip("bulk", async () => {
       //const connection = dummyConnection;
       //const connection = getLocalLibsqlConnection();
       const credentials = null as any; // @todo: determine what to do here

app/__test__/data/specs/WithBuilder.spec.ts

@@ -1,4 +1,5 @@
 import { afterAll, describe, expect, test } from "bun:test";
+import { _jsonp } from "../../../src/core/utils";
 import {
    Entity,
    EntityManager,
@@ -8,19 +9,56 @@ import {
    TextField,
    WithBuilder
 } from "../../../src/data";
+import * as proto from "../../../src/data/prototype";
+import { compileQb, prettyPrintQb, schemaToEm } from "../../helper";
 import { getDummyConnection } from "../helper";
 
-const { dummyConnection, afterAllCleanup } = getDummyConnection();
-afterAll(afterAllCleanup);
+const { dummyConnection } = getDummyConnection();
 
 describe("[data] WithBuilder", async () => {
+   test("validate withs", async () => {
+      const schema = proto.em(
+         {
+            posts: proto.entity("posts", {}),
+            users: proto.entity("users", {}),
+            media: proto.entity("media", {})
+         },
+         ({ relation }, { posts, users, media }) => {
+            relation(posts).manyToOne(users);
+            relation(users).polyToOne(media, { mappedBy: "avatar" });
+         }
+      );
+      const em = schemaToEm(schema);
+
+      expect(WithBuilder.validateWiths(em, "posts", undefined)).toBe(0);
+      expect(WithBuilder.validateWiths(em, "posts", {})).toBe(0);
+      expect(WithBuilder.validateWiths(em, "posts", { users: {} })).toBe(1);
+      expect(
+         WithBuilder.validateWiths(em, "posts", {
+            users: {
+               with: { avatar: {} }
+            }
+         })
+      ).toBe(2);
+      expect(() => WithBuilder.validateWiths(em, "posts", { author: {} })).toThrow();
+      expect(() =>
+         WithBuilder.validateWiths(em, "posts", {
+            users: {
+               with: { glibberish: {} }
+            }
+         })
+      ).toThrow();
+   });
+
    test("missing relation", async () => {
       const users = new Entity("users", [new TextField("username")]);
       const em = new EntityManager([users], dummyConnection);
 
       expect(() =>
-         WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, ["posts"])
-      ).toThrow('Relation "posts" not found');
+         WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, {
+            posts: {}
+         })
+      ).toThrow('Relation "users<>posts" not found');
    });
 
    test("addClause: ManyToOne", async () => {
@@ -29,36 +67,39 @@ describe("[data] WithBuilder", async () => {
       const relations = [new ManyToOneRelation(posts, users, { mappedBy: "author" })];
       const em = new EntityManager([users, posts], dummyConnection, relations);
 
-      const qb = WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, [
-         "posts"
-      ]);
+      const qb = WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, {
+         posts: {}
+      });
 
       const res = qb.compile();
 
       expect(res.sql).toBe(
-         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'content\', "agg"."content", \'author_id\', "agg"."author_id")), \'[]\') from (select "posts"."id" as "id", "posts"."content" as "content", "posts"."author_id" as "author_id" from "posts" where "users"."id" = "posts"."author_id" limit ?) as agg) as "posts" from "users"'
+         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'content\', "agg"."content", \'author_id\', "agg"."author_id")), \'[]\') from (select "posts"."id" as "id", "posts"."content" as "content", "posts"."author_id" as "author_id" from "posts" as "posts" where "posts"."author_id" = "users"."id" order by "posts"."id" asc limit ? offset ?) as agg) as "posts" from "users"'
       );
-      expect(res.parameters).toEqual([5]);
+      expect(res.parameters).toEqual([10, 0]);
 
       const qb2 = WithBuilder.addClause(
          em,
          em.connection.kysely.selectFrom("posts"),
          posts, // @todo: try with "users", it gives output!
-         ["author"]
+         {
+            author: {}
+         }
       );
 
       const res2 = qb2.compile();
 
       expect(res2.sql).toBe(
-         'select (select json_object(\'id\', "obj"."id", \'username\', "obj"."username") from (select "users"."id" as "id", "users"."username" as "username" from "users" where "posts"."author_id" = "users"."id" limit ?) as obj) as "author" from "posts"'
+         'select (select json_object(\'id\', "obj"."id", \'username\', "obj"."username") from (select "users"."id" as "id", "users"."username" as "username" from "users" as "author" where "author"."id" = "posts"."author_id" order by "users"."id" asc limit ? offset ?) as obj) as "author" from "posts"'
       );
-      expect(res2.parameters).toEqual([1]);
+      expect(res2.parameters).toEqual([1, 0]);
    });
 
    test("test with empty join", async () => {
+      const em = new EntityManager([], dummyConnection);
       const qb = { qb: 1 } as any;
 
-      expect(WithBuilder.addClause(null as any, qb, null as any, [])).toBe(qb);
+      expect(WithBuilder.addClause(em, qb, null as any, {})).toBe(qb);
    });
 
    test("test manytomany", async () => {
@@ -89,7 +130,7 @@ describe("[data] WithBuilder", async () => {
 
       //console.log((await em.repository().findMany("posts_categories")).result);
 
-      const res = await em.repository(posts).findMany({ with: ["categories"] });
+      const res = await em.repository(posts).findMany({ with: { categories: {} } });
 
       expect(res.data).toEqual([
          {
@@ -107,7 +148,7 @@ describe("[data] WithBuilder", async () => {
          }
       ]);
 
-      const res2 = await em.repository(categories).findMany({ with: ["posts"] });
+      const res2 = await em.repository(categories).findMany({ with: { posts: {} } });
 
       //console.log(res2.sql, res2.data);
 
@@ -121,8 +162,8 @@ describe("[data] WithBuilder", async () => {
             id: 2,
             label: "beauty",
             posts: [
-               { id: 2, title: "beauty post" },
-               { id: 1, title: "fashion post" }
+               { id: 1, title: "fashion post" },
+               { id: 2, title: "beauty post" }
             ]
          },
          {
@@ -150,25 +191,25 @@ describe("[data] WithBuilder", async () => {
          em,
          em.connection.kysely.selectFrom("categories"),
          categories,
-         ["single"]
+         { single: {} }
       );
       const res = qb.compile();
       expect(res.sql).toBe(
-         'select (select json_object(\'id\', "obj"."id", \'path\', "obj"."path") from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" limit ?) as obj) as "single" from "categories"'
+         'select (select json_object(\'id\', "obj"."id", \'path\', "obj"."path") from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" order by "media"."id" asc limit ? offset ?) as obj) as "single" from "categories"'
       );
-      expect(res.parameters).toEqual(["categories.single", 1]);
+      expect(res.parameters).toEqual(["categories.single", 1, 0]);
 
       const qb2 = WithBuilder.addClause(
          em,
          em.connection.kysely.selectFrom("categories"),
          categories,
-         ["multiple"]
+         { multiple: {} }
       );
       const res2 = qb2.compile();
       expect(res2.sql).toBe(
-         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'path\', "agg"."path")), \'[]\') from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" limit ?) as agg) as "multiple" from "categories"'
+         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'path\', "agg"."path")), \'[]\') from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" order by "media"."id" asc limit ? offset ?) as agg) as "multiple" from "categories"'
       );
-      expect(res2.parameters).toEqual(["categories.multiple", 5]);
+      expect(res2.parameters).toEqual(["categories.multiple", 10, 0]);
    });
 
    /*test("test manytoone", async () => {
@@ -192,4 +233,205 @@ describe("[data] WithBuilder", async () => {
       const res = await em.repository().findMany("posts", { join: ["author"] });
       console.log(res.sql, res.parameters, res.result);
    });*/
+
+   describe("recursive", () => {
+      test("compiles with singles", async () => {
+         const schema = proto.em(
+            {
+               posts: proto.entity("posts", {}),
+               users: proto.entity("users", {
+                  username: proto.text()
+               }),
+               media: proto.entity("media", {
+                  path: proto.text()
+               })
+            },
+            ({ relation }, { posts, users, media }) => {
+               relation(posts).manyToOne(users);
+               relation(users).polyToOne(media, { mappedBy: "avatar" });
+            }
+         );
+         const em = schemaToEm(schema);
+
+         const qb = WithBuilder.addClause(
+            em,
+            em.connection.kysely.selectFrom("posts"),
+            schema.entities.posts,
+            {
+               users: {
+                  limit: 5, // ignored
+                  select: ["id", "username"],
+                  sort: { by: "username", dir: "asc" },
+                  with: {
+                     avatar: {
+                        select: ["id", "path"],
+                        limit: 2 // ignored
+                     }
+                  }
+               }
+            }
+         );
+
+         //prettyPrintQb(qb);
+         expect(qb.compile().sql).toBe(
+            'select (select json_object(\'id\', "obj"."id", \'username\', "obj"."username", \'avatar\', "obj"."avatar") from (select "users"."id" as "id", "users"."username" as "username", (select json_object(\'id\', "obj"."id", \'path\', "obj"."path") from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "users"."id" = "media"."entity_id" order by "media"."id" asc limit ? offset ?) as obj) as "avatar" from "users" as "users" where "users"."id" = "posts"."users_id" order by "users"."username" asc limit ? offset ?) as obj) as "users" from "posts"'
+         );
+         expect(qb.compile().parameters).toEqual(["users.avatar", 1, 0, 1, 0]);
+      });
+
+      test("compiles with many", async () => {
+         const schema = proto.em(
+            {
+               posts: proto.entity("posts", {}),
+               comments: proto.entity("comments", {}),
+               users: proto.entity("users", {
+                  username: proto.text()
+               }),
+               media: proto.entity("media", {
+                  path: proto.text()
+               })
+            },
+            ({ relation }, { posts, comments, users, media }) => {
+               relation(posts).manyToOne(users).polyToOne(media, { mappedBy: "images" });
+               relation(users).polyToOne(media, { mappedBy: "avatar" });
+               relation(comments).manyToOne(posts).manyToOne(users);
+            }
+         );
+         const em = schemaToEm(schema);
+
+         const qb = WithBuilder.addClause(
+            em,
+            em.connection.kysely.selectFrom("posts"),
+            schema.entities.posts,
+            {
+               comments: {
+                  limit: 12,
+                  with: {
+                     users: {
+                        select: ["username"]
+                     }
+                  }
+               }
+            }
+         );
+
+         expect(qb.compile().sql).toBe(
+            'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'posts_id\', "agg"."posts_id", \'users_id\', "agg"."users_id", \'users\', "agg"."users")), \'[]\') from (select "comments"."id" as "id", "comments"."posts_id" as "posts_id", "comments"."users_id" as "users_id", (select json_object(\'username\', "obj"."username") from (select "users"."username" as "username" from "users" as "users" where "users"."id" = "comments"."users_id" order by "users"."id" asc limit ? offset ?) as obj) as "users" from "comments" as "comments" where "comments"."posts_id" = "posts"."id" order by "comments"."id" asc limit ? offset ?) as agg) as "comments" from "posts"'
+         );
+         expect(qb.compile().parameters).toEqual([1, 0, 12, 0]);
+      });
+
+      test("returns correct result", async () => {
+         const schema = proto.em(
+            {
+               posts: proto.entity("posts", {
+                  title: proto.text()
+               }),
+               comments: proto.entity("comments", {
+                  content: proto.text()
+               }),
+               users: proto.entity("users", {
+                  username: proto.text()
+               }),
+               media: proto.entity("media", {
+                  path: proto.text()
+               })
+            },
+            ({ relation }, { posts, comments, users, media }) => {
+               relation(posts).manyToOne(users).polyToOne(media, { mappedBy: "images" });
+               relation(users).polyToOne(media, { mappedBy: "avatar" });
+               relation(comments).manyToOne(posts).manyToOne(users);
+            }
+         );
+         const em = schemaToEm(schema);
+         await em.schema().sync({ force: true });
+
+         // add data
+         await em.mutator("users").insertMany([{ username: "user1" }, { username: "user2" }]);
+         await em.mutator("posts").insertMany([
+            { title: "post1", users_id: 1 },
+            { title: "post2", users_id: 1 },
+            { title: "post3", users_id: 2 }
+         ]);
+         await em.mutator("comments").insertMany([
+            { content: "comment1", posts_id: 1, users_id: 1 },
+            { content: "comment1-1", posts_id: 1, users_id: 1 },
+            { content: "comment2", posts_id: 1, users_id: 2 },
+            { content: "comment3", posts_id: 2, users_id: 1 },
+            { content: "comment4", posts_id: 2, users_id: 2 },
+            { content: "comment5", posts_id: 3, users_id: 1 },
+            { content: "comment6", posts_id: 3, users_id: 2 }
+         ]);
+
+         const result = await em.repo("posts").findMany({
+            select: ["title"],
+            with: {
+               comments: {
+                  limit: 2,
+                  select: ["content"],
+                  with: {
+                     users: {
+                        select: ["username"]
+                     }
+                  }
+               }
+            }
+         });
+
+         expect(result.data).toEqual([
+            {
+               title: "post1",
+               comments: [
+                  {
+                     content: "comment1",
+                     users: {
+                        username: "user1"
+                     }
+                  },
+                  {
+                     content: "comment1-1",
+                     users: {
+                        username: "user1"
+                     }
+                  }
+               ]
+            },
+            {
+               title: "post2",
+               comments: [
+                  {
+                     content: "comment3",
+                     users: {
+                        username: "user1"
+                     }
+                  },
+                  {
+                     content: "comment4",
+                     users: {
+                        username: "user2"
+                     }
+                  }
+               ]
+            },
+            {
+               title: "post3",
+               comments: [
+                  {
+                     content: "comment5",
+                     users: {
+                        username: "user1"
+                     }
+                  },
+                  {
+                     content: "comment6",
+                     users: {
+                        username: "user2"
+                     }
+                  }
+               ]
+            }
+         ]);
+         //console.log(_jsonp(result.data));
+      });
+   });
 });

app/__test__/data/specs/fields/EnumField.spec.ts

@@ -13,10 +13,6 @@ describe("[data] EnumField", async () => {
       { options: options(["a", "b", "c"]) }
    );
 
-   test("yields if no options", async () => {
-      expect(() => new EnumField("test", { options: options([]) })).toThrow();
-   });
-
    test("yields if default value is not a valid option", async () => {
       expect(
          () => new EnumField("test", { options: options(["a", "b"]), default_value: "c" })

app/__test__/data/specs/fields/Field.spec.ts

@@ -15,11 +15,9 @@ describe("[data] Field", async () => {
 
    runBaseFieldTests(FieldSpec, { defaultValue: "test", schemaType: "text" });
 
-   test.only("default config", async () => {
-      const field = new FieldSpec("test");
+   test("default config", async () => {
       const config = Default(baseFieldConfigSchema, {});
       expect(stripMark(new FieldSpec("test").config)).toEqual(config);
-      console.log("config", new TextField("test", { required: true }).toJSON());
    });
 
    test("transformPersist (specific)", async () => {

app/__test__/data/specs/fields/JsonField.spec.ts

@@ -32,7 +32,7 @@ describe("[data] JsonField", async () => {
    });
 
    test("getValue", async () => {
-      expect(field.getValue({ test: 1 }, "form")).toBe('{"test":1}');
+      expect(field.getValue({ test: 1 }, "form")).toBe('{\n  "test": 1\n}');
       expect(field.getValue("string", "form")).toBe('"string"');
       expect(field.getValue(1, "form")).toBe("1");
 

app/__test__/data/specs/relations/EntityRelation.spec.ts

@@ -70,9 +70,9 @@ describe("[data] EntityRelation", async () => {
 
    it("required", async () => {
       const relation1 = new TestEntityRelation();
-      expect(relation1.config.required).toBe(false);
+      expect(relation1.required).toBe(false);
 
       const relation2 = new TestEntityRelation({ required: true });
-      expect(relation2.config.required).toBe(true);
+      expect(relation2.required).toBe(true);
    });
 });

app/__test__/flows/workflow-basic.test.ts

@@ -1,7 +1,7 @@
 // eslint-disable-next-line import/no-unresolved
 import { describe, expect, test } from "bun:test";
 import { isEqual } from "lodash-es";
-import { type Static, Type, _jsonp } from "../../src/core/utils";
+import { type Static, Type, _jsonp, withDisabledConsole } from "../../src/core/utils";
 import { Condition, ExecutionEvent, FetchTask, Flow, LogTask, Task } from "../../src/flows";
 
 /*beforeAll(disableConsoleLog);
@@ -232,7 +232,9 @@ describe("Flow tests", async () => {
       ).toEqual(["second", "fourth"]);
 
       const execution = back.createExecution();
-      expect(execution.start()).rejects.toThrow();
+      withDisabledConsole(async () => {
+         expect(execution.start()).rejects.toThrow();
+      });
    });
 
    test("Flow with back step: enough retries", async () => {

app/__test__/helper.ts

@@ -1,7 +1,9 @@
 import { unlink } from "node:fs/promises";
-import type { SqliteDatabase } from "kysely";
+import type { SelectQueryBuilder, SqliteDatabase } from "kysely";
 import Database from "libsql";
-import { SqliteLocalConnection } from "../src/data";
+import { format as sqlFormat } from "sql-formatter";
+import { type Connection, EntityManager, SqliteLocalConnection } from "../src/data";
+import type { em as protoEm } from "../src/data/prototype";
 
 export function getDummyDatabase(memory: boolean = true): {
    dummyDb: SqliteDatabase;
@@ -40,7 +42,7 @@ const _oldConsoles = {
    error: console.error
 };
 
-export function disableConsoleLog(severities: ConsoleSeverity[] = ["log"]) {
+export function disableConsoleLog(severities: ConsoleSeverity[] = ["log", "warn"]) {
    severities.forEach((severity) => {
       console[severity] = () => null;
    });
@@ -51,3 +53,18 @@ export function enableConsoleLog() {
       console[severity as ConsoleSeverity] = fn;
    });
 }
+
+export function compileQb(qb: SelectQueryBuilder<any, any, any>) {
+   const { sql, parameters } = qb.compile();
+   return { sql, parameters };
+}
+
+export function prettyPrintQb(qb: SelectQueryBuilder<any, any, any>) {
+   const { sql, parameters } = qb.compile();
+   console.log("$", sqlFormat(sql), "\n[params]", parameters);
+}
+
+export function schemaToEm(s: ReturnType<typeof protoEm>, conn?: Connection): EntityManager<any> {
+   const connection = conn ? conn : getDummyConnection().dummyConnection;
+   return new EntityManager(Object.values(s.entities), connection, s.relations, s.indices);
+}

app/__test__/integration/auth.integration.test.ts

@@ -0,0 +1,213 @@
+import { afterAll, beforeAll, describe, expect, it } from "bun:test";
+import { App, createApp } from "../../src";
+import type { AuthResponse } from "../../src/auth";
+import { randomString, secureRandomString, withDisabledConsole } from "../../src/core/utils";
+import { disableConsoleLog, enableConsoleLog } from "../helper";
+
+beforeAll(disableConsoleLog);
+afterAll(enableConsoleLog);
+
+const roles = {
+   sloppy: {
+      guest: {
+         permissions: [
+            "system.access.admin",
+            "system.schema.read",
+            "system.access.api",
+            "system.config.read",
+            "data.entity.read"
+         ],
+         is_default: true
+      },
+      admin: {
+         is_default: true,
+         implicit_allow: true
+      }
+   },
+   strict: {
+      guest: {
+         permissions: ["system.access.api", "system.config.read", "data.entity.read"],
+         is_default: true
+      },
+      admin: {
+         is_default: true,
+         implicit_allow: true
+      }
+   }
+};
+const configs = {
+   auth: {
+      enabled: true,
+      entity_name: "users",
+      jwt: {
+         secret: secureRandomString(20),
+         issuer: randomString(10)
+      },
+      roles: roles.strict,
+      guard: {
+         enabled: true
+      }
+   },
+   users: {
+      normal: {
+         email: "normal@bknd.io",
+         password: "12345678"
+      },
+      admin: {
+         email: "admin@bknd.io",
+         password: "12345678",
+         role: "admin"
+      }
+   }
+};
+
+function createAuthApp() {
+   const app = createApp({
+      initialConfig: {
+         auth: configs.auth
+      }
+   });
+
+   app.emgr.onEvent(
+      App.Events.AppFirstBoot,
+      async () => {
+         await app.createUser(configs.users.normal);
+         await app.createUser(configs.users.admin);
+      },
+      "sync"
+   );
+
+   return app;
+}
+
+function getCookie(r: Response, name: string) {
+   const cookies = r.headers.get("cookie") ?? r.headers.get("set-cookie");
+   if (!cookies) return;
+   const cookie = cookies.split(";").find((c) => c.trim().startsWith(name));
+   if (!cookie) return;
+   return cookie.split("=")[1];
+}
+
+const fns = <Mode extends "cookie" | "token" = "token">(app: App, mode?: Mode) => {
+   function headers(token?: string, additional?: Record<string, string>) {
+      if (mode === "cookie") {
+         return {
+            cookie: `auth=${token};`,
+            ...additional
+         };
+      }
+
+      return {
+         Authorization: `Bearer ${token}`,
+         "Content-Type": "application/json",
+         ...additional
+      };
+   }
+   function body(obj?: Record<string, any>) {
+      if (mode === "cookie") {
+         const formData = new FormData();
+         for (const key in obj) {
+            formData.append(key, obj[key]);
+         }
+         return formData;
+      }
+
+      return JSON.stringify(obj);
+   }
+
+   return {
+      login: async (
+         user: any
+      ): Promise<{ res: Response; data: Mode extends "token" ? AuthResponse : string }> => {
+         const res = (await app.server.request("/api/auth/password/login", {
+            method: "POST",
+            headers: headers(),
+            body: body(user)
+         })) as Response;
+
+         const data = mode === "cookie" ? getCookie(res, "auth") : await res.json();
+
+         return { res, data };
+      },
+      me: async (token?: string): Promise<Pick<AuthResponse, "user">> => {
+         const res = (await app.server.request("/api/auth/me", {
+            method: "GET",
+            headers: headers(token)
+         })) as Response;
+         return await res.json();
+      }
+   };
+};
+
+describe("integration auth", () => {
+   it("should create users on boot", async () => {
+      const app = createAuthApp();
+      await app.build();
+
+      const { data: users } = await app.em.repository("users").findMany();
+      expect(users.length).toBe(2);
+      expect(users[0].email).toBe(configs.users.normal.email);
+      expect(users[1].email).toBe(configs.users.admin.email);
+   });
+
+   it("should log you in with API", async () => {
+      const app = createAuthApp();
+      await app.build();
+      const $fns = fns(app);
+
+      // login api
+      const { data } = await $fns.login(configs.users.normal);
+      const me = await $fns.me(data.token);
+
+      expect(data.user.email).toBe(me.user.email);
+      expect(me.user.email).toBe(configs.users.normal.email);
+
+      // expect no user with no token
+      expect(await $fns.me()).toEqual({ user: null as any });
+
+      // expect no user with invalid token
+      expect(await $fns.me("invalid")).toEqual({ user: null as any });
+      expect(await $fns.me()).toEqual({ user: null as any });
+   });
+
+   it("should log you in with form and cookie", async () => {
+      const app = createAuthApp();
+      await app.build();
+      const $fns = fns(app, "cookie");
+
+      const { res, data: token } = await $fns.login(configs.users.normal);
+      expect(token).toBeDefined();
+      expect(res.status).toBe(302); // because it redirects
+
+      // test cookie jwt interchangability
+      {
+         // expect token to not work as-is for api endpoints
+         expect(await fns(app).me(token)).toEqual({ user: null as any });
+         // hono adds an additional segment to cookies
+         const apified_token = token.split(".").slice(0, -1).join(".");
+         // now it should work
+         // @todo: maybe add a config to don't allow re-use?
+         expect((await fns(app).me(apified_token)).user.email).toBe(configs.users.normal.email);
+      }
+
+      // test cookie with me endpoint
+      {
+         const me = await $fns.me(token);
+         expect(me.user.email).toBe(configs.users.normal.email);
+
+         // check with invalid & empty
+         expect(await $fns.me("invalid")).toEqual({ user: null as any });
+         expect(await $fns.me()).toEqual({ user: null as any });
+      }
+   });
+
+   it("should check for permissions", async () => {
+      const app = createAuthApp();
+      await app.build();
+
+      await withDisabledConsole(async () => {
+         const res = await app.server.request("/api/system/schema");
+         expect(res.status).toBe(403);
+      });
+   });
+});

app/__test__/media/adapters/StorageCloudinaryAdapter.spec.ts

@@ -3,7 +3,7 @@ import { randomString } from "../../../src/core/utils";
 import { StorageCloudinaryAdapter } from "../../../src/media";
 
 import { config } from "dotenv";
-const dotenvOutput = config({ path: `${import.meta.dir}/../../.env` });
+const dotenvOutput = config({ path: `${import.meta.dir}/../../../.env` });
 const {
    CLOUDINARY_CLOUD_NAME,
    CLOUDINARY_API_KEY,

app/__test__/media/adapters/StorageLocalAdapter.spec.ts

@@ -15,7 +15,7 @@ describe("StorageLocalAdapter", () => {
 
    test("puts an object", async () => {
       objects = (await adapter.listObjects()).length;
-      expect(await adapter.putObject(filename, await file.arrayBuffer())).toBeString();
+      expect(await adapter.putObject(filename, file)).toBeString();
    });
 
    test("lists objects", async () => {

app/__test__/media/adapters/StorageS3Adapter.spec.ts

@@ -3,14 +3,14 @@ import { randomString } from "../../../src/core/utils";
 import { StorageS3Adapter } from "../../../src/media";
 
 import { config } from "dotenv";
-const dotenvOutput = config({ path: `${import.meta.dir}/../../.env` });
+const dotenvOutput = config({ path: `${import.meta.dir}/../../../.env` });
 const { R2_ACCESS_KEY, R2_SECRET_ACCESS_KEY, R2_URL, AWS_ACCESS_KEY, AWS_SECRET_KEY, AWS_S3_URL } =
    dotenvOutput.parsed!;
 
 // @todo: mock r2/s3 responses for faster tests
-const ALL_TESTS = process.env.ALL_TESTS;
+const ALL_TESTS = !!process.env.ALL_TESTS;
 
-describe("Storage", async () => {
+describe.skipIf(ALL_TESTS)("StorageS3Adapter", async () => {
    console.log("ALL_TESTS", process.env.ALL_TESTS);
    const versions = [
       [

app/__test__/modules/AppAuth.spec.ts

@@ -1,5 +1,7 @@
-import { afterAll, beforeAll, beforeEach, describe, expect, test } from "bun:test";
+import { afterAll, beforeAll, beforeEach, describe, expect, spyOn, test } from "bun:test";
+import { createApp } from "../../src";
 import { AuthController } from "../../src/auth/api/AuthController";
+import { em, entity, make, text } from "../../src/data";
 import { AppAuth, type ModuleBuildContext } from "../../src/modules";
 import { disableConsoleLog, enableConsoleLog } from "../helper";
 import { makeCtx, moduleTestSuite } from "./module-test-suite";
@@ -76,4 +78,87 @@ describe("AppAuth", () => {
          expect(users[0].email).toBe("some@body.com");
       }
    });
+
+   test("registers auth middleware for bknd routes only", async () => {
+      const app = createApp({
+         initialConfig: {
+            auth: {
+               enabled: true,
+               jwt: {
+                  secret: "123456"
+               }
+            }
+         }
+      });
+
+      await app.build();
+      const spy = spyOn(app.module.auth.authenticator, "requestCookieRefresh");
+
+      // register custom route
+      app.server.get("/test", async (c) => c.text("test"));
+
+      // call a system api and then the custom route
+      await app.server.request("/api/system/ping");
+      await app.server.request("/test");
+
+      expect(spy.mock.calls.length).toBe(1);
+   });
+
+   test("should allow additional user fields", async () => {
+      const app = createApp({
+         initialConfig: {
+            auth: {
+               entity_name: "users",
+               enabled: true
+            },
+            data: em({
+               users: entity("users", {
+                  additional: text()
+               })
+            }).toJSON()
+         }
+      });
+
+      await app.build();
+
+      const e = app.modules.em.entity("users");
+      const fields = e.fields.map((f) => f.name);
+      expect(e.type).toBe("system");
+      expect(fields).toContain("additional");
+      expect(fields).toEqual(["id", "additional", "email", "strategy", "strategy_value", "role"]);
+   });
+
+   test("ensure user field configs is always correct", async () => {
+      const app = createApp({
+         initialConfig: {
+            auth: {
+               enabled: true
+            },
+            data: em({
+               users: entity("users", {
+                  strategy: text({
+                     fillable: true,
+                     hidden: false
+                  }),
+                  strategy_value: text({
+                     fillable: true,
+                     hidden: false
+                  })
+               })
+            }).toJSON()
+         }
+      });
+      await app.build();
+
+      const users = app.em.entity("users");
+      const props = ["hidden", "fillable", "required"];
+
+      for (const [name, _authFieldProto] of Object.entries(AppAuth.usersFields)) {
+         const authField = make(name, _authFieldProto as any);
+         const field = users.field(name)!;
+         for (const prop of props) {
+            expect(field.config[prop]).toBe(authField.config[prop]);
+         }
+      }
+   });
 });

app/__test__/modules/AppMedia.spec.ts

@@ -1,7 +1,55 @@
-import { describe } from "bun:test";
+import { describe, expect, test } from "bun:test";
+import { createApp, registries } from "../../src";
+import { em, entity, text } from "../../src/data";
+import { StorageLocalAdapter } from "../../src/media/storage/adapters/StorageLocalAdapter";
 import { AppMedia } from "../../src/modules";
 import { moduleTestSuite } from "./module-test-suite";
 
 describe("AppMedia", () => {
    moduleTestSuite(AppMedia);
+
+   test("should allow additional fields", async () => {
+      registries.media.register("local", StorageLocalAdapter);
+
+      const app = createApp({
+         initialConfig: {
+            media: {
+               entity_name: "media",
+               enabled: true,
+               adapter: {
+                  type: "local",
+                  config: {
+                     path: "./"
+                  }
+               }
+            },
+            data: em({
+               media: entity("media", {
+                  additional: text()
+               })
+            }).toJSON()
+         }
+      });
+
+      await app.build();
+
+      const e = app.modules.em.entity("media");
+      const fields = e.fields.map((f) => f.name);
+      expect(e.type).toBe("system");
+      expect(fields).toContain("additional");
+      expect(fields).toEqual([
+         "id",
+         "additional",
+         "path",
+         "folder",
+         "mime_type",
+         "size",
+         "scope",
+         "etag",
+         "modified_at",
+         "reference",
+         "entity_id",
+         "metadata"
+      ]);
+   });
 });

app/__test__/modules/Module.spec.ts

@@ -0,0 +1,212 @@
+import { describe, expect, test } from "bun:test";
+import { type TSchema, Type, stripMark } from "../../src/core/utils";
+import { EntityManager, em, entity, index, text } from "../../src/data";
+import { DummyConnection } from "../../src/data/connection/DummyConnection";
+import { Module } from "../../src/modules/Module";
+
+function createModule<Schema extends TSchema>(schema: Schema) {
+   class TestModule extends Module<typeof schema> {
+      getSchema() {
+         return schema;
+      }
+      toJSON() {
+         return this.config;
+      }
+      useForceParse() {
+         return true;
+      }
+   }
+
+   return TestModule;
+}
+
+describe("Module", async () => {
+   describe("basic", () => {
+      test("listener", async () => {
+         let result: any;
+
+         const module = createModule(Type.Object({ a: Type.String() }));
+         const m = new module({ a: "test" });
+
+         await m.schema().set({ a: "test2" });
+         m.setListener(async (c) => {
+            await new Promise((r) => setTimeout(r, 10));
+            result = stripMark(c);
+         });
+         await m.schema().set({ a: "test3" });
+         expect(result).toEqual({ a: "test3" });
+      });
+   });
+
+   describe("db schema", () => {
+      class M extends Module {
+         override getSchema() {
+            return Type.Object({});
+         }
+
+         prt = {
+            ensureEntity: this.ensureEntity.bind(this),
+            ensureIndex: this.ensureIndex.bind(this),
+            ensureSchema: this.ensureSchema.bind(this)
+         };
+
+         get em() {
+            return this.ctx.em;
+         }
+      }
+
+      function make(_em: ReturnType<typeof em>) {
+         const em = new EntityManager(
+            Object.values(_em.entities),
+            new DummyConnection(),
+            _em.relations,
+            _em.indices
+         );
+         return new M({} as any, { em, flags: Module.ctx_flags } as any);
+      }
+      function flat(_em: EntityManager) {
+         return {
+            entities: _em.entities.map((e) => ({
+               name: e.name,
+               fields: e.fields.map((f) => f.name),
+               type: e.type
+            })),
+            indices: _em.indices.map((i) => ({
+               name: i.name,
+               entity: i.entity.name,
+               fields: i.fields.map((f) => f.name),
+               unique: i.unique
+            }))
+         };
+      }
+
+      test("no change", () => {
+         const initial = em({});
+
+         const m = make(initial);
+         expect(m.ctx.flags.sync_required).toBe(false);
+
+         expect(flat(make(initial).em)).toEqual({
+            entities: [],
+            indices: []
+         });
+      });
+
+      test("init", () => {
+         const initial = em({
+            users: entity("u", {
+               name: text()
+            })
+         });
+
+         const m = make(initial);
+         expect(m.ctx.flags.sync_required).toBe(false);
+
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name"],
+                  type: "regular"
+               }
+            ],
+            indices: []
+         });
+      });
+
+      test("ensure entity", () => {
+         const initial = em({
+            users: entity("u", {
+               name: text()
+            })
+         });
+
+         const m = make(initial);
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name"],
+                  type: "regular"
+               }
+            ],
+            indices: []
+         });
+
+         // this should add a new entity
+         m.prt.ensureEntity(
+            entity("p", {
+               title: text()
+            })
+         );
+
+         // this should only add the field "important"
+         m.prt.ensureEntity(
+            entity(
+               "u",
+               {
+                  important: text()
+               },
+               undefined,
+               "system"
+            )
+         );
+
+         expect(m.ctx.flags.sync_required).toBe(true);
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name", "important"],
+                  // ensured type must be present
+                  type: "system"
+               },
+               {
+                  name: "p",
+                  fields: ["id", "title"],
+                  type: "regular"
+               }
+            ],
+            indices: []
+         });
+      });
+
+      test("ensure index", () => {
+         const users = entity("u", {
+            name: text(),
+            title: text()
+         });
+         const initial = em({ users }, ({ index }, { users }) => {
+            index(users).on(["title"]);
+         });
+
+         const m = make(initial);
+         m.prt.ensureIndex(index(users).on(["name"]));
+
+         expect(m.ctx.flags.sync_required).toBe(true);
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name", "title"],
+                  type: "regular"
+               }
+            ],
+            indices: [
+               {
+                  name: "idx_u_title",
+                  entity: "u",
+                  fields: ["title"],
+                  unique: false
+               },
+               {
+                  name: "idx_u_name",
+                  entity: "u",
+                  fields: ["name"],
+                  unique: false
+               }
+            ]
+         });
+      });
+   });
+});

app/__test__/modules/ModuleManager.spec.ts

@@ -1,9 +1,9 @@
 import { describe, expect, test } from "bun:test";
-import { mark, stripMark } from "../src/core/utils";
-import { entity, text } from "../src/data";
-import { ModuleManager, getDefaultConfig } from "../src/modules/ModuleManager";
-import { CURRENT_VERSION, TABLE_NAME } from "../src/modules/migrations";
-import { getDummyConnection } from "./helper";
+import { stripMark } from "../../src/core/utils";
+import { entity, text } from "../../src/data";
+import { ModuleManager, getDefaultConfig } from "../../src/modules/ModuleManager";
+import { CURRENT_VERSION, TABLE_NAME } from "../../src/modules/migrations";
+import { getDummyConnection } from "../helper";
 
 describe("ModuleManager", async () => {
    test("s1: no config, no build", async () => {

app/__test__/modules/module-test-suite.ts

@@ -2,10 +2,11 @@ import { beforeEach, describe, expect, it } from "bun:test";
 
 import { Hono } from "hono";
 import { Guard } from "../../src/auth";
+import { DebugLogger } from "../../src/core";
 import { EventManager } from "../../src/core/events";
 import { Default, stripMark } from "../../src/core/utils";
 import { EntityManager } from "../../src/data";
-import type { Module, ModuleBuildContext } from "../../src/modules/Module";
+import { Module, type ModuleBuildContext } from "../../src/modules/Module";
 import { getDummyConnection } from "../helper";
 
 export function makeCtx(overrides?: Partial<ModuleBuildContext>): ModuleBuildContext {
@@ -16,6 +17,8 @@ export function makeCtx(overrides?: Partial<ModuleBuildContext>): ModuleBuildCon
       em: new EntityManager([], dummyConnection),
       emgr: new EventManager(),
       guard: new Guard(),
+      flags: Module.ctx_flags,
+      logger: new DebugLogger(false),
       ...overrides
    };
 }

app/build.ts

@@ -1,8 +1,5 @@
 import { $ } from "bun";
-import * as esbuild from "esbuild";
-import postcss from "esbuild-postcss";
 import * as tsup from "tsup";
-import { guessMimeType } from "./src/media/storage/mime-types";
 
 const args = process.argv.slice(2);
 const watch = args.includes("--watch");
@@ -12,19 +9,21 @@ const sourcemap = args.includes("--sourcemap");
 const clean = args.includes("--clean");
 
 if (clean) {
-   console.log("Cleaning dist");
-   await $`rm -rf dist`;
+   console.log("Cleaning dist (w/o static)");
+   await $`find dist -mindepth 1 ! -path "dist/static/*" ! -path "dist/static" -exec rm -rf {} +`;
 }
 
 let types_running = false;
 function buildTypes() {
-   if (types_running) return;
+   if (types_running || !types) return;
    types_running = true;
 
    Bun.spawn(["bun", "build:types"], {
+      stdout: "inherit",
       onExit: () => {
          console.log("Types built");
          Bun.spawn(["bun", "tsc-alias"], {
+            stdout: "inherit",
             onExit: () => {
                console.log("Types aliased");
                types_running = false;
@@ -36,7 +35,7 @@ function buildTypes() {
 
 let watcher_timeout: any;
 function delayTypes() {
-   if (!watch) return;
+   if (!watch || !types) return;
    if (watcher_timeout) {
       clearTimeout(watcher_timeout);
    }
@@ -47,67 +46,6 @@ if (types && !watch) {
    buildTypes();
 }
 
-/**
- * Build static assets
- * Using esbuild because tsup doesn't include "react"
- */
-const result = await esbuild.build({
-   minify,
-   sourcemap,
-   entryPoints: ["src/ui/main.tsx"],
-   entryNames: "[dir]/[name]-[hash]",
-   outdir: "dist/static",
-   platform: "browser",
-   bundle: true,
-   splitting: true,
-   metafile: true,
-   drop: ["console", "debugger"],
-   inject: ["src/ui/inject.js"],
-   target: "es2022",
-   format: "esm",
-   plugins: [postcss()],
-   loader: {
-      ".svg": "dataurl",
-      ".js": "jsx"
-   },
-   define: {
-      __isDev: "0",
-      "process.env.NODE_ENV": '"production"'
-   },
-   chunkNames: "chunks/[name]-[hash]",
-   logLevel: "error"
-});
-
-// Write manifest
-{
-   const manifest: Record<string, object> = {};
-   const toAsset = (output: string) => {
-      const name = output.split("/").pop()!;
-      return {
-         name,
-         path: output,
-         mime: guessMimeType(name)
-      };
-   };
-
-   const info = Object.entries(result.metafile.outputs)
-      .filter(([, meta]) => {
-         return meta.entryPoint && meta.entryPoint === "src/ui/main.tsx";
-      })
-      .map(([output, meta]) => ({ output, meta }));
-
-   for (const { output, meta } of info) {
-      manifest[meta.entryPoint as string] = toAsset(output);
-      if (meta.cssBundle) {
-         manifest["src/ui/main.css"] = toAsset(meta.cssBundle);
-      }
-   }
-
-   const manifest_file = "dist/static/manifest.json";
-   await Bun.write(manifest_file, JSON.stringify(manifest, null, 2));
-   console.log(`Manifest written to ${manifest_file}`, manifest);
-}
-
 /**
  * Building backend and general API
  */
@@ -120,7 +58,7 @@ await tsup.build({
    external: ["bun:test", "@libsql/client"],
    metafile: true,
    platform: "browser",
-   format: ["esm", "cjs"],
+   format: ["esm"],
    splitting: false,
    treeshake: true,
    loader: {
@@ -138,26 +76,80 @@ await tsup.build({
    minify,
    sourcemap,
    watch,
-   entry: ["src/ui/index.ts", "src/ui/client/index.ts", "src/ui/main.css"],
+   entry: ["src/ui/index.ts", "src/ui/client/index.ts", "src/ui/main.css", "src/ui/styles.css"],
    outDir: "dist/ui",
-   external: ["bun:test", "react", "react-dom", "use-sync-external-store"],
+   external: [
+      "bun:test",
+      "react",
+      "react-dom",
+      "react/jsx-runtime",
+      "react/jsx-dev-runtime",
+      "use-sync-external-store",
+      /codemirror/,
+      "@xyflow/react",
+      "@mantine/core"
+   ],
    metafile: true,
    platform: "browser",
-   format: ["esm", "cjs"],
-   splitting: true,
+   format: ["esm"],
+   splitting: false,
+   bundle: true,
    treeshake: true,
    loader: {
       ".svg": "dataurl"
    },
    esbuildOptions: (options) => {
       options.logLevel = "silent";
-      options.chunkNames = "chunks/[name]-[hash]";
    },
    onSuccess: async () => {
       delayTypes();
    }
 });
 
+/**
+ * Building UI Elements
+ * - tailwind-merge is mocked, no exclude
+ * - ui/client is external, and after built replaced with "bknd/client"
+ */
+await tsup.build({
+   minify,
+   sourcemap,
+   watch,
+   entry: ["src/ui/elements/index.ts"],
+   outDir: "dist/ui/elements",
+   external: [
+      "ui/client",
+      "react",
+      "react-dom",
+      "react/jsx-runtime",
+      "react/jsx-dev-runtime",
+      "use-sync-external-store"
+   ],
+   metafile: true,
+   platform: "browser",
+   format: ["esm"],
+   splitting: false,
+   bundle: true,
+   treeshake: true,
+   loader: {
+      ".svg": "dataurl"
+   },
+   esbuildOptions: (options) => {
+      options.alias = {
+         // not important for elements, mock to reduce bundle
+         "tailwind-merge": "./src/ui/elements/mocks/tailwind-merge.ts"
+      };
+   },
+   onSuccess: async () => {
+      // manually replace ui/client with bknd/client
+      const path = "./dist/ui/elements/index.js";
+      const bundle = await Bun.file(path).text();
+      await Bun.write(path, bundle.replaceAll("ui/client", "bknd/client"));
+
+      delayTypes();
+   }
+});
+
 /**
  * Building adapters
  */
@@ -166,7 +158,7 @@ function baseConfig(adapter: string): tsup.Options {
       minify,
       sourcemap,
       watch,
-      entry: [`src/adapter/${adapter}`],
+      entry: [`src/adapter/${adapter}/index.ts`],
       format: ["esm"],
       platform: "neutral",
       outDir: `dist/adapter/${adapter}`,
@@ -188,37 +180,22 @@ function baseConfig(adapter: string): tsup.Options {
    };
 }
 
+await tsup.build(baseConfig("remix"));
+await tsup.build(baseConfig("bun"));
+await tsup.build(baseConfig("astro"));
+await tsup.build(baseConfig("cloudflare"));
+
 await tsup.build({
    ...baseConfig("vite"),
    platform: "node"
 });
 
-await tsup.build({
-   ...baseConfig("cloudflare")
-});
-
 await tsup.build({
    ...baseConfig("nextjs"),
-   format: ["esm", "cjs"],
    platform: "node"
 });
 
-await tsup.build({
-   ...baseConfig("remix"),
-   format: ["esm", "cjs"]
-});
-
-await tsup.build({
-   ...baseConfig("bun")
-});
-
 await tsup.build({
    ...baseConfig("node"),
-   platform: "node",
-   format: ["esm", "cjs"]
-});
-
-await tsup.build({
-   ...baseConfig("astro"),
-   format: ["esm", "cjs"]
+   platform: "node"
 });

app/bunfig.toml

@@ -1,2 +1,5 @@
 [install]
-registry = "http://localhost:4873"
+#registry = "http://localhost:4873"
+
+[test]
+coverageSkipTestFiles = true

app/package.json

@@ -3,22 +3,32 @@
   "type": "module",
   "sideEffects": false,
   "bin": "./dist/cli/index.js",
-  "version": "0.4.0",
+  "version": "0.6.2",
+  "description": "Lightweight Firebase/Supabase alternative built to run anywhere — incl. Next.js, Remix, Astro, Cloudflare, Bun, Node, AWS Lambda & more.",
+  "homepage": "https://bknd.io",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/bknd-io/bknd.git"
+  },
+  "bugs": {
+    "url": "https://github.com/bknd-io/bknd/issues"
+  },
   "scripts": {
-    "build:all": "NODE_ENV=production bun run build.ts --minify --types --clean && bun run build:cli",
     "dev": "vite",
     "test": "ALL_TESTS=1 bun test --bail",
+    "test:coverage": "ALL_TESTS=1 bun test --bail --coverage",
     "build": "NODE_ENV=production bun run build.ts --minify --types",
+    "build:all": "rm -rf dist && bun run build:static && NODE_ENV=production bun run build.ts --minify --types --clean && bun run build:cli",
+    "build:cli": "bun build src/cli/index.ts --target node --outdir dist/cli --minify",
+    "build:static": "vite build",
     "watch": "bun run build.ts --types --watch",
     "types": "bun tsc --noEmit",
     "clean:types": "find ./dist -name '*.d.ts' -delete && rm -f ./dist/tsconfig.tsbuildinfo",
     "build:types": "tsc --emitDeclarationOnly && tsc-alias",
-    "build:css": "bun tailwindcss -i src/ui/main.css -o ./dist/static/styles.css",
-    "watch:css": "bun tailwindcss --watch -i src/ui/main.css -o ./dist/styles.css",
     "updater": "bun x npm-check-updates -ui",
-    "build:cli": "bun build src/cli/index.ts --target node --outdir dist/cli --minify",
     "cli": "LOCAL=1 bun src/cli/index.ts",
-    "prepublishOnly": "bun run build:all"
+    "prepublishOnly": "bun run types && bun run test && bun run build:all && cp ../README.md ./",
+    "postpublish": "rm -f README.md"
   },
   "license": "FSL-1.1-MIT",
   "dependencies": {
@@ -34,34 +44,33 @@
     "liquidjs": "^10.15.0",
     "lodash-es": "^4.17.21",
     "oauth4webapi": "^2.11.1",
-    "swr": "^2.2.5"
-  },
-  "devDependencies": {
-    "@aws-sdk/client-s3": "^3.613.0",
+    "swr": "^2.2.5",
+    "json-schema-form-react": "^0.0.2",
+    "@uiw/react-codemirror": "^4.23.6",
     "@codemirror/lang-html": "^6.4.9",
     "@codemirror/lang-json": "^6.0.1",
     "@codemirror/lang-liquid": "^6.2.1",
+    "@xyflow/react": "^12.3.2",
+    "@mantine/core": "^7.13.4",
+    "@hello-pangea/dnd": "^17.0.0"
+  },
+  "devDependencies": {
+    "@aws-sdk/client-s3": "^3.613.0",
     "@dagrejs/dagre": "^1.1.4",
-    "@hello-pangea/dnd": "^17.0.0",
     "@hono/typebox-validator": "^0.2.6",
     "@hono/vite-dev-server": "^0.17.0",
     "@hono/zod-validator": "^0.4.1",
     "@hookform/resolvers": "^3.9.1",
     "@libsql/kysely-libsql": "^0.4.1",
-    "@mantine/core": "^7.13.4",
-    "@mantine/hooks": "^7.13.4",
-    "@mantine/modals": "^7.13.4",
-    "@mantine/notifications": "^7.13.5",
     "@radix-ui/react-scroll-area": "^1.2.0",
     "@rjsf/core": "^5.22.2",
     "@tabler/icons-react": "3.18.0",
     "@types/node": "^22.10.0",
     "@types/react": "^18.3.12",
     "@types/react-dom": "^18.3.1",
-    "@uiw/react-codemirror": "^4.23.6",
     "@vitejs/plugin-react": "^4.3.3",
-    "@xyflow/react": "^12.3.2",
     "autoprefixer": "^10.4.20",
+    "clsx": "^2.1.1",
     "esbuild-postcss": "^0.0.4",
     "jotai": "^2.10.1",
     "open": "^10.1.0",
@@ -72,6 +81,7 @@
     "react-hook-form": "^7.53.1",
     "react-icons": "5.2.1",
     "react-json-view-lite": "^2.0.1",
+    "sql-formatter": "^15.4.9",
     "tailwind-merge": "^2.5.4",
     "tailwindcss": "^3.4.14",
     "tailwindcss-animate": "^1.0.7",
@@ -103,6 +113,11 @@
       "import": "./dist/ui/index.js",
       "require": "./dist/ui/index.cjs"
     },
+    "./elements": {
+      "types": "./dist/types/ui/elements/index.d.ts",
+      "import": "./dist/ui/elements/index.js",
+      "require": "./dist/ui/elements/index.cjs"
+    },
     "./client": {
       "types": "./dist/types/ui/client/index.d.ts",
       "import": "./dist/ui/client/index.js",
@@ -163,8 +178,9 @@
       "import": "./dist/adapter/astro/index.js",
       "require": "./dist/adapter/astro/index.cjs"
     },
-    "./dist/styles.css": "./dist/ui/main.css",
-    "./dist/manifest.json": "./dist/static/manifest.json"
+    "./dist/main.css": "./dist/ui/main.css",
+    "./dist/styles.css": "./dist/ui/styles.css",
+    "./dist/manifest.json": "./dist/static/.vite/manifest.json"
   },
   "publishConfig": {
     "access": "public"
@@ -177,5 +193,21 @@
     "!dist/**/*.map",
     "!dist/metafile*",
     "!dist/**/metafile*"
+  ],
+  "keywords": [
+    "backend",
+    "database",
+    "authentication",
+    "media",
+    "workflows",
+    "api",
+    "jwt",
+    "serverless",
+    "cloudflare",
+    "nextjs",
+    "remix",
+    "astro",
+    "bun",
+    "node"
   ]
 }

app/src/Api.ts

@@ -128,15 +128,17 @@ export class Api {
       };
    }
 
-   async getVerifiedAuthState(force?: boolean): Promise<AuthState> {
-      if (force === true || !this.verified) {
-         await this.verifyAuth();
-      }
-
+   async getVerifiedAuthState(): Promise<AuthState> {
+      await this.verifyAuth();
       return this.getAuthState();
    }
 
    async verifyAuth() {
+      if (!this.token) {
+         this.markAuthVerified(false);
+         return;
+      }
+
       try {
          const res = await this.auth.me();
          if (!res.ok || !res.body.user) {

app/src/App.ts

@@ -1,3 +1,4 @@
+import type { CreateUserPayload } from "auth/AppAuth";
 import { Event } from "core/events";
 import { Connection, type LibSqlCredentials, LibsqlConnection } from "data";
 import {
@@ -68,6 +69,12 @@ export class App {
          onFirstBoot: async () => {
             console.log("[APP] first boot");
             this.trigger_first_boot = true;
+         },
+         onServerInit: async (server) => {
+            server.use(async (c, next) => {
+               c.set("app", this);
+               await next();
+            });
          }
       });
       this.modules.ctx().emgr.registerEvents(AppEvents);
@@ -87,20 +94,20 @@ export class App {
          //console.log("syncing", syncResult);
       }
 
+      const { guard, server } = this.modules.ctx();
+
       // load system controller
-      this.modules.ctx().guard.registerPermissions(Object.values(SystemPermissions));
-      this.modules.server.route("/api/system", new SystemController(this).getController());
+      guard.registerPermissions(Object.values(SystemPermissions));
+      server.route("/api/system", new SystemController(this).getController());
 
       // load plugins
       if (this.plugins.length > 0) {
          await Promise.all(this.plugins.map((plugin) => plugin(this)));
       }
 
-      //console.log("emitting built", options);
       await this.emgr.emit(new AppBuiltEvent({ app: this }));
 
-      // not found on any not registered api route
-      this.modules.server.all("/api/*", async (c) => c.notFound());
+      server.all("/api/*", async (c) => c.notFound());
 
       if (options?.save) {
          await this.modules.save();
@@ -121,6 +128,10 @@ export class App {
       return this.modules.server;
    }
 
+   get em() {
+      return this.modules.ctx().em;
+   }
+
    get fetch(): any {
       return this.server.fetch;
    }
@@ -147,7 +158,7 @@ export class App {
    registerAdminController(config?: AdminControllerOptions) {
       // register admin
       this.adminController = new AdminController(this, config);
-      this.modules.server.route("/", this.adminController.getController());
+      this.modules.server.route(config?.basepath ?? "/", this.adminController.getController());
       return this;
    }
 
@@ -158,6 +169,10 @@ export class App {
    static create(config: CreateAppConfig) {
       return createApp(config);
    }
+
+   async createUser(p: CreateUserPayload) {
+      return this.module.auth.createUser(p);
+   }
 }
 
 export function createApp(config: CreateAppConfig = {}) {

app/src/adapter/bun/bun.adapter.ts

@@ -11,13 +11,7 @@ let app: App;
 
 export type BunBkndConfig = RuntimeBkndConfig & Omit<ServeOptions, "fetch">;
 
-export async function createApp({
-   distPath,
-   onBuilt,
-   buildConfig,
-   beforeBuild,
-   ...config
-}: RuntimeBkndConfig = {}) {
+export async function createApp({ distPath, ...config }: RuntimeBkndConfig = {}) {
    const root = path.resolve(distPath ?? "./node_modules/bknd/dist", "static");
 
    if (!app) {

app/src/adapter/index.ts

@@ -1,5 +1,6 @@
 import type { IncomingMessage } from "node:http";
 import { App, type CreateAppConfig, registries } from "bknd";
+import { config as $config } from "core";
 import type { MiddlewareHandler } from "hono";
 import { StorageLocalAdapter } from "media/storage/adapters/StorageLocalAdapter";
 import type { AdminControllerOptions } from "modules/server/AdminController";
@@ -106,12 +107,10 @@ export async function createRuntimeApp<Env = any>(
       App.Events.AppBuiltEvent,
       async () => {
          if (serveStatic) {
-            if (Array.isArray(serveStatic)) {
-               const [path, handler] = serveStatic;
-               app.modules.server.get(path, handler);
-            } else {
-               app.modules.server.get("/*", serveStatic);
-            }
+            const [path, handler] = Array.isArray(serveStatic)
+               ? serveStatic
+               : [$config.server.assets_path + "*", serveStatic];
+            app.modules.server.get(path, handler);
          }
 
          await config.onBuilt?.(app);

app/src/adapter/nextjs/nextjs.adapter.ts

@@ -2,7 +2,9 @@ import type { IncomingMessage, ServerResponse } from "node:http";
 import { Api, type App } from "bknd";
 import { type FrameworkBkndConfig, createFrameworkApp, nodeRequestToRequest } from "../index";
 
-export type NextjsBkndConfig = FrameworkBkndConfig;
+export type NextjsBkndConfig = FrameworkBkndConfig & {
+   cleanSearch?: string[];
+};
 
 type GetServerSidePropsContext = {
    req: IncomingMessage;
@@ -32,10 +34,13 @@ export function withApi<T>(handler: (ctx: GetServerSidePropsContext & { api: Api
    };
 }
 
-function getCleanRequest(req: Request) {
-   // clean search params from "route" attribute
+function getCleanRequest(
+   req: Request,
+   { cleanSearch = ["route"] }: Pick<NextjsBkndConfig, "cleanSearch">
+) {
    const url = new URL(req.url);
-   url.searchParams.delete("route");
+   cleanSearch?.forEach((k) => url.searchParams.delete(k));
+
    return new Request(url.toString(), {
       method: req.method,
       headers: req.headers,
@@ -44,12 +49,12 @@ function getCleanRequest(req: Request) {
 }
 
 let app: App;
-export function serve(config: NextjsBkndConfig = {}) {
+export function serve({ cleanSearch, ...config }: NextjsBkndConfig = {}) {
    return async (req: Request) => {
       if (!app) {
          app = await createFrameworkApp(config);
       }
-      const request = getCleanRequest(req);
+      const request = getCleanRequest(req, { cleanSearch });
       return app.fetch(request, process.env);
    };
 }

app/src/adapter/node/node.adapter.ts

@@ -19,9 +19,6 @@ export function serve({
    port = $config.server.default_port,
    hostname,
    listener,
-   onBuilt,
-   buildConfig = {},
-   beforeBuild,
    ...config
 }: NodeBkndConfig = {}) {
    const root = path.relative(

app/src/adapter/vite/dev-server-config.ts

@@ -0,0 +1,14 @@
+export const devServerConfig = {
+   entry: "./server.ts",
+   exclude: [
+      /.*\.tsx?($|\?)/,
+      /^(?!.*\/__admin).*\.(s?css|less)($|\?)/,
+      // exclude except /api
+      /^(?!.*\/api).*\.(ico|mp4|jpg|jpeg|svg|png|vtt|mp3|js)($|\?)/,
+      /^\/@.+$/,
+      /\/components.*?\.json.*/, // @todo: improve
+      /^\/(public|assets|static)\/.+/,
+      /^\/node_modules\/.*/
+   ] as any,
+   injectClientScript: false
+} as const;

app/src/adapter/vite/vite.adapter.ts

@@ -1,10 +1,13 @@
 import { serveStatic } from "@hono/node-server/serve-static";
+import { type DevServerOptions, default as honoViteDevServer } from "@hono/vite-dev-server";
 import { type RuntimeBkndConfig, createRuntimeApp } from "adapter";
 import type { App } from "bknd";
+import { devServerConfig } from "./dev-server-config";
 
 export type ViteBkndConfig<Env = any> = RuntimeBkndConfig<Env> & {
+   mode?: "cached" | "fresh";
    setAdminHtml?: boolean;
-   forceDev?: boolean;
+   forceDev?: boolean | { mainPath: string };
    html?: string;
 };
 
@@ -24,20 +27,27 @@ ${addBkndContext ? "<!-- BKND_CONTEXT -->" : ""}
    );
 }
 
-async function createApp(config: ViteBkndConfig, env?: any) {
+async function createApp(config: ViteBkndConfig = {}, env?: any) {
    return await createRuntimeApp(
       {
          ...config,
-         adminOptions: config.setAdminHtml
-            ? { html: config.html, forceDev: config.forceDev }
-            : undefined,
+         registerLocalMedia: true,
+         adminOptions:
+            config.setAdminHtml === false
+               ? undefined
+               : {
+                    html: config.html,
+                    forceDev: config.forceDev ?? {
+                       mainPath: "/src/main.tsx"
+                    }
+                 },
          serveStatic: ["/assets/*", serveStatic({ root: config.distPath ?? "./" })]
       },
       env
    );
 }
 
-export async function serveFresh(config: ViteBkndConfig) {
+export function serveFresh(config: Omit<ViteBkndConfig, "mode"> = {}) {
    return {
       async fetch(request: Request, env: any, ctx: ExecutionContext) {
          const app = await createApp(config, env);
@@ -47,7 +57,7 @@ export async function serveFresh(config: ViteBkndConfig) {
 }
 
 let app: App;
-export async function serveCached(config: ViteBkndConfig) {
+export function serveCached(config: Omit<ViteBkndConfig, "mode"> = {}) {
    return {
       async fetch(request: Request, env: any, ctx: ExecutionContext) {
          if (!app) {
@@ -58,3 +68,14 @@ export async function serveCached(config: ViteBkndConfig) {
       }
    };
 }
+
+export function serve({ mode, ...config }: ViteBkndConfig = {}) {
+   return mode === "fresh" ? serveFresh(config) : serveCached(config);
+}
+
+export function devServer(options: DevServerOptions) {
+   return honoViteDevServer({
+      ...devServerConfig,
+      ...options
+   });
+}

app/src/auth/AppAuth.ts

@@ -1,9 +1,16 @@
-import { type AuthAction, Authenticator, type ProfileExchange, Role, type Strategy } from "auth";
+import {
+   type AuthAction,
+   AuthPermissions,
+   Authenticator,
+   type ProfileExchange,
+   Role,
+   type Strategy
+} from "auth";
 import type { PasswordStrategy } from "auth/authenticate/strategies";
-import { Exception, type PrimaryFieldType } from "core";
+import { type DB, Exception, type PrimaryFieldType } from "core";
 import { type Static, secureRandomString, transformObject } from "core/utils";
-import { type Entity, EntityIndex, type EntityManager } from "data";
-import { type FieldSchema, entity, enumm, make, text } from "data/prototype";
+import type { Entity, EntityManager } from "data";
+import { type FieldSchema, em, entity, enumm, text } from "data/prototype";
 import { pick } from "lodash-es";
 import { Module } from "modules/Module";
 import { AuthController } from "./api/AuthController";
@@ -17,6 +24,7 @@ declare module "core" {
 }
 
 type AuthSchema = Static<typeof authConfigSchema>;
+export type CreateUserPayload = { email: string; password: string; [key: string]: any };
 
 export class AppAuth extends Module<typeof authConfigSchema> {
    private _authenticator?: Authenticator;
@@ -36,8 +44,12 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       return to;
    }
 
+   get enabled() {
+      return this.config.enabled;
+   }
+
    override async build() {
-      if (!this.config.enabled) {
+      if (!this.enabled) {
          this.setBuilt();
          return;
       }
@@ -72,8 +84,8 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       super.setBuilt();
 
       this._controller = new AuthController(this);
-      //this.ctx.server.use(controller.getMiddleware);
       this.ctx.server.route(this.config.basepath, this._controller.getController());
+      this.ctx.guard.registerPermissions(Object.values(AuthPermissions));
    }
 
    get controller(): AuthController {
@@ -84,14 +96,6 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       return this._controller;
    }
 
-   getMiddleware() {
-      if (!this.config.enabled) {
-         return;
-      }
-
-      return new AuthController(this).getMiddleware;
-   }
-
    getSchema() {
       return authConfigSchema;
    }
@@ -111,12 +115,12 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       identifier: string,
       profile: ProfileExchange
    ): Promise<any> {
-      console.log("***** AppAuth:resolveUser", {
+      /*console.log("***** AppAuth:resolveUser", {
          action,
          strategy: strategy.getName(),
          identifier,
          profile
-      });
+      });*/
       if (!this.config.allow_register && action === "register") {
          throw new Exception("Registration is not allowed", 403);
       }
@@ -137,12 +141,12 @@ export class AppAuth extends Module<typeof authConfigSchema> {
    }
 
    private filterUserData(user: any) {
-      console.log(
+      /*console.log(
          "--filterUserData",
          user,
          this.config.jwt.fields,
          pick(user, this.config.jwt.fields)
-      );
+      );*/
       return pick(user, this.config.jwt.fields);
    }
 
@@ -168,18 +172,18 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       if (!result.data) {
          throw new Exception("User not found", 404);
       }
-      console.log("---login data", result.data, result);
+      //console.log("---login data", result.data, result);
 
       // compare strategy and identifier
-      console.log("strategy comparison", result.data.strategy, strategy.getName());
+      //console.log("strategy comparison", result.data.strategy, strategy.getName());
       if (result.data.strategy !== strategy.getName()) {
-         console.log("!!! User registered with different strategy");
+         //console.log("!!! User registered with different strategy");
          throw new Exception("User registered with different strategy");
       }
 
-      console.log("identifier comparison", result.data.strategy_value, identifier);
+      //console.log("identifier comparison", result.data.strategy_value, identifier);
       if (result.data.strategy_value !== identifier) {
-         console.log("!!! Invalid credentials");
+         //console.log("!!! Invalid credentials");
          throw new Exception("Invalid credentials");
       }
 
@@ -220,10 +224,23 @@ export class AppAuth extends Module<typeof authConfigSchema> {
    }
 
    private toggleStrategyValueVisibility(visible: boolean) {
-      const field = this.getUsersEntity().field("strategy_value")!;
+      const toggle = (name: string, visible: boolean) => {
+         const field = this.getUsersEntity().field(name)!;
+
+         if (visible) {
+            field.config.hidden = false;
+            field.config.fillable = true;
+         } else {
+            // reset to normal
+            const template = AppAuth.usersFields.strategy_value.config;
+            field.config.hidden = template.hidden;
+            field.config.fillable = template.fillable;
+         }
+      };
+
+      toggle("strategy_value", visible);
+      toggle("strategy", visible);
 
-      field.config.hidden = !visible;
-      field.config.fillable = visible;
       // @todo: think about a PasswordField that automatically hashes on save?
    }
 
@@ -238,7 +255,10 @@ export class AppAuth extends Module<typeof authConfigSchema> {
 
    static usersFields = {
       email: text().required(),
-      strategy: text({ fillable: ["create"], hidden: ["form"] }).required(),
+      strategy: text({
+         fillable: ["create"],
+         hidden: ["update", "form"]
+      }).required(),
       strategy_value: text({
          fillable: ["create"],
          hidden: ["read", "table", "update", "form"]
@@ -247,51 +267,34 @@ export class AppAuth extends Module<typeof authConfigSchema> {
    };
 
    registerEntities() {
-      const users = this.getUsersEntity();
-
-      if (!this.em.hasEntity(users.name)) {
-         this.em.addEntity(users);
-      } else {
-         // if exists, check all fields required are there
-         // @todo: add to context: "needs sync" flag
-         const _entity = this.getUsersEntity(true);
-         for (const field of _entity.fields) {
-            const _field = users.field(field.name);
-            if (!_field) {
-               users.addField(field);
+      const users = this.getUsersEntity(true);
+      this.ensureSchema(
+         em(
+            {
+               [users.name as "users"]: users
+            },
+            ({ index }, { users }) => {
+               index(users).on(["email"], true).on(["strategy"]).on(["strategy_value"]);
             }
-         }
-      }
-
-      const indices = [
-         new EntityIndex(users, [users.field("email")!], true),
-         new EntityIndex(users, [users.field("strategy")!]),
-         new EntityIndex(users, [users.field("strategy_value")!])
-      ];
-      indices.forEach((index) => {
-         if (!this.em.hasIndex(index)) {
-            this.em.addIndex(index);
-         }
-      });
+         )
+      );
 
       try {
          const roles = Object.keys(this.config.roles ?? {});
-         const field = make("role", enumm({ enum: roles }));
-         this.em.entity(users.name).__experimental_replaceField("role", field);
+         this.replaceEntityField(users, "role", enumm({ enum: roles }));
       } catch (e) {}
 
       try {
          const strategies = Object.keys(this.config.strategies ?? {});
-         const field = make("strategy", enumm({ enum: strategies }));
-         this.em.entity(users.name).__experimental_replaceField("strategy", field);
+         this.replaceEntityField(users, "strategy", enumm({ enum: strategies }));
       } catch (e) {}
    }
 
-   async createUser({
-      email,
-      password,
-      ...additional
-   }: { email: string; password: string; [key: string]: any }) {
+   async createUser({ email, password, ...additional }: CreateUserPayload): Promise<DB["users"]> {
+      if (!this.enabled) {
+         throw new Error("Cannot create user, auth not enabled");
+      }
+
       const strategy = "password";
       const pw = this.authenticator.strategy(strategy) as PasswordStrategy;
       const strategy_value = await pw.hash(password);

app/src/auth/api/AuthApi.ts

@@ -1,4 +1,5 @@
-import type { AppAuthSchema, AppAuthStrategies } from "auth/auth-schema";
+import type { AuthActionResponse } from "auth/api/AuthController";
+import type { AppAuthSchema } from "auth/auth-schema";
 import type { AuthResponse, SafeUser, Strategy } from "auth/authenticate/Authenticator";
 import { type BaseModuleApiOptions, ModuleApi } from "modules/ModuleApi";
 
@@ -13,22 +14,46 @@ export class AuthApi extends ModuleApi<AuthApiOptions> {
       };
    }
 
-   async loginWithPassword(input: any) {
-      const res = await this.post<AuthResponse>(["password", "login"], input);
+   async login(strategy: string, input: any) {
+      const res = await this.post<AuthResponse>([strategy, "login"], input);
       if (res.ok && res.body.token) {
          await this.options.onTokenUpdate?.(res.body.token);
       }
       return res;
    }
 
-   async registerWithPassword(input: any) {
-      const res = await this.post<AuthResponse>(["password", "register"], input);
+   async register(strategy: string, input: any) {
+      const res = await this.post<AuthResponse>([strategy, "register"], input);
       if (res.ok && res.body.token) {
          await this.options.onTokenUpdate?.(res.body.token);
       }
       return res;
    }
 
+   async actionSchema(strategy: string, action: string) {
+      return this.get<Strategy>([strategy, "actions", action, "schema.json"]);
+   }
+
+   async action(strategy: string, action: string, input: any) {
+      return this.post<AuthActionResponse>([strategy, "actions", action], input);
+   }
+
+   /**
+    * @deprecated use login("password", ...) instead
+    * @param input
+    */
+   async loginWithPassword(input: any) {
+      return this.login("password", input);
+   }
+
+   /**
+    * @deprecated use register("password", ...) instead
+    * @param input
+    */
+   async registerWithPassword(input: any) {
+      return this.register("password", input);
+   }
+
    me() {
       return this.get<{ user: SafeUser | null }>(["me"]);
    }

app/src/auth/api/AuthController.ts

@@ -1,58 +1,108 @@
-import type { AppAuth } from "auth";
-import { type ClassController, isDebug } from "core";
-import { Hono, type MiddlewareHandler } from "hono";
+import { type AppAuth, AuthPermissions, type SafeUser, type Strategy } from "auth";
+import { TypeInvalidError, parse } from "core/utils";
+import { DataPermissions } from "data";
+import type { Hono } from "hono";
+import { Controller } from "modules/Controller";
+import type { ServerEnv } from "modules/Module";
 
-export class AuthController implements ClassController {
-   constructor(private auth: AppAuth) {}
+export type AuthActionResponse = {
+   success: boolean;
+   action: string;
+   data?: SafeUser;
+   errors?: any;
+};
+
+export class AuthController extends Controller {
+   constructor(private auth: AppAuth) {
+      super();
+   }
 
    get guard() {
       return this.auth.ctx.guard;
    }
 
-   getMiddleware: MiddlewareHandler = async (c, next) => {
-      // @todo: ONLY HOTFIX
-      // middlewares are added for all routes are registered. But we need to make sure that
-      // only HTML/JSON routes are adding a cookie to the response. Config updates might
-      // also use an extension "syntax", e.g. /api/system/patch/data/entities.posts
-      // This middleware should be extracted and added by each Controller individually,
-      // but it requires access to the auth secret.
-      // Note: This doesn't mean endpoints aren't protected, just the cookie is not set.
-      const url = new URL(c.req.url);
-      const last = url.pathname.split("/")?.pop();
-      const ext = last?.includes(".") ? last.split(".")?.pop() : undefined;
-      if (
-         !this.auth.authenticator.isJsonRequest(c) &&
-         ["GET", "HEAD", "OPTIONS"].includes(c.req.method) &&
-         ext &&
-         ["js", "css", "png", "jpg", "jpeg", "svg", "ico"].includes(ext)
-      ) {
-         isDebug() && console.log("Skipping auth", { ext }, url.pathname);
-      } else {
-         const user = await this.auth.authenticator.resolveAuthFromRequest(c);
-         this.auth.ctx.guard.setUserContext(user);
+   private registerStrategyActions(strategy: Strategy, mainHono: Hono<ServerEnv>) {
+      const actions = strategy.getActions?.();
+      if (!actions) {
+         return;
       }
 
-      await next();
-   };
+      const { auth, permission } = this.middlewares;
+      const hono = this.create().use(auth());
 
-   getController(): Hono<any> {
-      const hono = new Hono();
+      const name = strategy.getName();
+      const { create, change } = actions;
+      const em = this.auth.em;
+
+      if (create) {
+         hono.post(
+            "/create",
+            permission([AuthPermissions.createUser, DataPermissions.entityCreate]),
+            async (c) => {
+               try {
+                  const body = await this.auth.authenticator.getBody(c);
+                  const valid = parse(create.schema, body, {
+                     skipMark: true
+                  });
+                  const processed = (await create.preprocess?.(valid)) ?? valid;
+
+                  // @todo: check processed for "role" and check permissions
+                  const mutator = em.mutator(this.auth.config.entity_name as "users");
+                  mutator.__unstable_toggleSystemEntityCreation(false);
+                  const { data: created } = await mutator.insertOne({
+                     ...processed,
+                     strategy: name
+                  });
+                  mutator.__unstable_toggleSystemEntityCreation(true);
+
+                  return c.json({
+                     success: true,
+                     action: "create",
+                     strategy: name,
+                     data: created as unknown as SafeUser
+                  } as AuthActionResponse);
+               } catch (e) {
+                  if (e instanceof TypeInvalidError) {
+                     return c.json(
+                        {
+                           success: false,
+                           errors: e.errors
+                        },
+                        400
+                     );
+                  }
+                  throw e;
+               }
+            }
+         );
+         hono.get("create/schema.json", async (c) => {
+            return c.json(create.schema);
+         });
+      }
+
+      mainHono.route(`/${name}/actions`, hono);
+   }
+
+   override getController() {
+      const { auth } = this.middlewares;
+      const hono = this.create();
       const strategies = this.auth.authenticator.getStrategies();
 
       for (const [name, strategy] of Object.entries(strategies)) {
          //console.log("registering", name, "at", `/${name}`);
          hono.route(`/${name}`, strategy.getController(this.auth.authenticator));
+         this.registerStrategyActions(strategy, hono);
       }
 
-      hono.get("/me", async (c) => {
+      hono.get("/me", auth(), async (c) => {
          if (this.auth.authenticator.isUserLoggedIn()) {
-            return c.json({ user: await this.auth.authenticator.getUser() });
+            return c.json({ user: this.auth.authenticator.getUser() });
          }
 
          return c.json({ user: null }, 403);
       });
 
-      hono.get("/logout", async (c) => {
+      hono.get("/logout", auth(), async (c) => {
          await this.auth.authenticator.logout(c);
          if (this.auth.authenticator.isJsonRequest(c)) {
             return c.json({ ok: true });

app/src/auth/auth-permissions.ts

@@ -0,0 +1,4 @@
+import { Permission } from "core";
+
+export const createUser = new Permission("auth.user.create");
+//export const updateUser = new Permission("auth.user.update");

app/src/auth/auth-schema.ts

@@ -33,6 +33,7 @@ const strategiesSchemaObject = objectTransform(STRATEGIES, (strategy, name) => {
 const strategiesSchema = Type.Union(Object.values(strategiesSchemaObject));
 export type AppAuthStrategies = Static<typeof strategiesSchema>;
 export type AppAuthOAuthStrategy = Static<typeof STRATEGIES.oauth.schema>;
+export type AppAuthCustomOAuthStrategy = Static<typeof STRATEGIES.custom_oauth.schema>;
 
 const guardConfigSchema = Type.Object({
    enabled: Type.Optional(Type.Boolean({ default: false }))

app/src/auth/authenticate/Authenticator.ts

@@ -1,23 +1,31 @@
-import { Exception } from "core";
+import { type DB, Exception } from "core";
 import { addFlashMessage } from "core/server/flash";
 import {
    type Static,
    StringEnum,
-   type TSchema,
+   type TObject,
    Type,
    parse,
-   randomString,
+   runtimeSupports,
    transformObject
 } from "core/utils";
 import type { Context, Hono } from "hono";
 import { deleteCookie, getSignedCookie, setSignedCookie } from "hono/cookie";
 import { sign, verify } from "hono/jwt";
 import type { CookieOptions } from "hono/utils/cookie";
-import { omit } from "lodash-es";
+import type { ServerEnv } from "modules/Module";
 
 type Input = any; // workaround
 export type JWTPayload = Parameters<typeof sign>[0];
 
+export const strategyActions = ["create", "change"] as const;
+export type StrategyActionName = (typeof strategyActions)[number];
+export type StrategyAction<S extends TObject = TObject> = {
+   schema: S;
+   preprocess: (input: unknown) => Promise<Omit<DB["users"], "id" | "strategy">>;
+};
+export type StrategyActions = Partial<Record<StrategyActionName, StrategyAction>>;
+
 // @todo: add schema to interface to ensure proper inference
 export interface Strategy {
    getController: (auth: Authenticator) => Hono<any>;
@@ -25,6 +33,7 @@ export interface Strategy {
    getMode: () => "form" | "external";
    getName: () => string;
    toJSON: (secrets?: boolean) => any;
+   getActions?: () => StrategyActions;
 }
 
 export type User = {
@@ -67,6 +76,9 @@ export const cookieConfig = Type.Partial(
    { default: {}, additionalProperties: false }
 );
 
+// @todo: maybe add a config to not allow cookie/api tokens to be used interchangably?
+// see auth.integration test for further details
+
 export const jwtConfig = Type.Object(
    {
       // @todo: autogenerate a secret if not present. But it must be persisted from AppAuth
@@ -98,7 +110,13 @@ export type AuthUserResolver = (
 export class Authenticator<Strategies extends Record<string, Strategy> = Record<string, Strategy>> {
    private readonly strategies: Strategies;
    private readonly config: AuthConfig;
-   private _user: SafeUser | undefined;
+   private _claims:
+      | undefined
+      | (SafeUser & {
+           iat: number;
+           iss?: string;
+           exp?: number;
+        });
    private readonly userResolver: AuthUserResolver;
 
    constructor(strategies: Strategies, userResolver?: AuthUserResolver, config?: AuthConfig) {
@@ -131,16 +149,18 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
    }
 
    isUserLoggedIn(): boolean {
-      return this._user !== undefined;
+      return this._claims !== undefined;
    }
 
-   getUser() {
-      return this._user;
+   getUser(): SafeUser | undefined {
+      if (!this._claims) return;
+
+      const { iat, exp, iss, ...user } = this._claims;
+      return user;
    }
 
-   // @todo: determine what to do exactly
-   __setUserNull() {
-      this._user = undefined;
+   resetUser() {
+      this._claims = undefined;
    }
 
    strategy<
@@ -154,6 +174,7 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       }
    }
 
+   // @todo: add jwt tests
    async jwt(user: Omit<User, "password">): Promise<string> {
       const prohibited = ["password"];
       for (const prop of prohibited) {
@@ -200,11 +221,11 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
             }
          }
 
-         this._user = omit(payload, ["iat", "exp", "iss"]) as SafeUser;
+         this._claims = payload as any;
          return true;
       } catch (e) {
-         this._user = undefined;
-         console.error(e);
+         this.resetUser();
+         //console.error(e);
       }
 
       return false;
@@ -222,10 +243,8 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
    private async getAuthCookie(c: Context): Promise<string | undefined> {
       try {
          const secret = this.config.jwt.secret;
-
          const token = await getSignedCookie(c, secret, "auth");
          if (typeof token !== "string") {
-            await deleteCookie(c, "auth", this.cookieOptions);
             return undefined;
          }
 
@@ -243,23 +262,27 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       if (this.config.cookie.renew) {
          const token = await this.getAuthCookie(c);
          if (token) {
-            console.log("renewing cookie", c.req.url);
             await this.setAuthCookie(c, token);
          }
       }
    }
 
-   private async setAuthCookie(c: Context, token: string) {
+   private async setAuthCookie(c: Context<ServerEnv>, token: string) {
       const secret = this.config.jwt.secret;
       await setSignedCookie(c, "auth", token, secret, this.cookieOptions);
    }
 
+   private async deleteAuthCookie(c: Context) {
+      await deleteCookie(c, "auth", this.cookieOptions);
+   }
+
    async logout(c: Context) {
       const cookie = await this.getAuthCookie(c);
       if (cookie) {
-         await deleteCookie(c, "auth", this.cookieOptions);
+         await this.deleteAuthCookie(c);
          await addFlashMessage(c, "Signed out", "info");
       }
+      this.resetUser();
    }
 
    // @todo: move this to a server helper
@@ -268,18 +291,39 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       return c.req.header("Content-Type") === "application/json";
    }
 
+   async getBody(c: Context) {
+      if (this.isJsonRequest(c)) {
+         return await c.req.json();
+      } else {
+         return Object.fromEntries((await c.req.formData()).entries());
+      }
+   }
+
+   private getSuccessPath(c: Context) {
+      const p = (this.config.cookie.pathSuccess ?? "/").replace(/\/+$/, "/");
+
+      // nextjs doesn't support non-fq urls
+      // but env could be proxied (stackblitz), so we shouldn't fq every url
+      if (!runtimeSupports("redirects_non_fq")) {
+         return new URL(c.req.url).origin + p;
+      }
+
+      return p;
+   }
+
    async respond(c: Context, data: AuthResponse | Error | any, redirect?: string) {
       if (this.isJsonRequest(c)) {
          return c.json(data);
       }
 
-      const successPath = this.config.cookie.pathSuccess ?? "/";
-      const successUrl = new URL(c.req.url).origin + successPath.replace(/\/+$/, "/");
-      const referer = new URL(redirect ?? c.req.header("Referer") ?? successUrl);
+      const successUrl = this.getSuccessPath(c);
+      const referer = redirect ?? c.req.header("Referer") ?? successUrl;
+      //console.log("auth respond", { redirect, successUrl, successPath });
 
       if ("token" in data) {
          await this.setAuthCookie(c, data.token);
          // can't navigate to "/" – doesn't work on nextjs
+         //console.log("auth success, redirecting to", successUrl);
          return c.redirect(successUrl);
       }
 
@@ -289,6 +333,7 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       }
 
       await addFlashMessage(c, message, "error");
+      //console.log("auth failed, redirecting to", referer);
       return c.redirect(referer);
    }
 
@@ -304,7 +349,7 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
 
       if (token) {
          await this.verify(token);
-         return this._user;
+         return this.getUser();
       }
 
       return undefined;
@@ -318,3 +363,13 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       };
    }
 }
+
+export function createStrategyAction<S extends TObject>(
+   schema: S,
+   preprocess: (input: Static<S>) => Promise<Partial<DB["users"]>>
+) {
+   return {
+      schema,
+      preprocess
+   } as StrategyAction<S>;
+}

app/src/auth/authenticate/strategies/PasswordStrategy.ts

@@ -2,6 +2,7 @@ import type { Authenticator, Strategy } from "auth";
 import { type Static, StringEnum, Type, parse } from "core/utils";
 import { hash } from "core/utils";
 import { type Context, Hono } from "hono";
+import { type StrategyAction, type StrategyActions, createStrategyAction } from "../Authenticator";
 
 type LoginSchema = { username: string; password: string } | { email: string; password: string };
 type RegisterSchema = { email: string; password: string; [key: string]: any };
@@ -54,17 +55,9 @@ export class PasswordStrategy implements Strategy {
    getController(authenticator: Authenticator): Hono<any> {
       const hono = new Hono();
 
-      async function getBody(c: Context) {
-         if (authenticator.isJsonRequest(c)) {
-            return await c.req.json();
-         } else {
-            return Object.fromEntries((await c.req.formData()).entries());
-         }
-      }
-
       return hono
          .post("/login", async (c) => {
-            const body = await getBody(c);
+            const body = await authenticator.getBody(c);
 
             try {
                const payload = await this.login(body);
@@ -76,7 +69,7 @@ export class PasswordStrategy implements Strategy {
             }
          })
          .post("/register", async (c) => {
-            const body = await getBody(c);
+            const body = await authenticator.getBody(c);
 
             const payload = await this.register(body);
             const data = await authenticator.resolve("register", this, payload.password, payload);
@@ -85,6 +78,27 @@ export class PasswordStrategy implements Strategy {
          });
    }
 
+   getActions(): StrategyActions {
+      return {
+         create: createStrategyAction(
+            Type.Object({
+               email: Type.String({
+                  pattern: "^[\\w-\\.]+@([\\w-]+\\.)+[\\w-]{2,4}$"
+               }),
+               password: Type.String({
+                  minLength: 8 // @todo: this should be configurable
+               })
+            }),
+            async ({ password, ...input }) => {
+               return {
+                  ...input,
+                  strategy_value: await this.hash(password)
+               };
+            }
+         )
+      };
+   }
+
    getSchema() {
       return schema;
    }

app/src/auth/authorize/Guard.ts

@@ -98,12 +98,16 @@ export class Guard {
       if (this.user && typeof this.user.role === "string") {
          const role = this.roles?.find((role) => role.name === this.user?.role);
          if (role) {
-            debug && console.log("guard: role found", this.user.role);
+            debug && console.log("guard: role found", [this.user.role]);
             return role;
          }
       }
 
-      debug && console.log("guard: role not found", this.user, this.user?.role);
+      debug &&
+         console.log("guard: role not found", {
+            user: this.user,
+            role: this.user?.role
+         });
       return this.getDefaultRole();
    }
 

app/src/auth/index.ts

@@ -19,3 +19,5 @@ export { AppAuth, type UserFieldSchema } from "./AppAuth";
 
 export { Guard, type GuardUserContext, type GuardConfig } from "./authorize/Guard";
 export { Role } from "./authorize/Role";
+
+export * as AuthPermissions from "./auth-permissions";

app/src/auth/middlewares.ts

@@ -0,0 +1,108 @@
+import type { Permission } from "core";
+import { patternMatch } from "core/utils";
+import type { Context } from "hono";
+import { createMiddleware } from "hono/factory";
+import type { ServerEnv } from "modules/Module";
+
+function getPath(reqOrCtx: Request | Context) {
+   const req = reqOrCtx instanceof Request ? reqOrCtx : reqOrCtx.req.raw;
+   return new URL(req.url).pathname;
+}
+
+export function shouldSkip(c: Context<ServerEnv>, skip?: (string | RegExp)[]) {
+   if (c.get("auth_skip")) return true;
+
+   const req = c.req.raw;
+   if (!skip) return false;
+
+   const path = getPath(req);
+   const result = skip.some((s) => patternMatch(path, s));
+
+   c.set("auth_skip", result);
+   return result;
+}
+
+export const auth = (options?: {
+   skip?: (string | RegExp)[];
+}) =>
+   createMiddleware<ServerEnv>(async (c, next) => {
+      const app = c.get("app");
+      const guard = app?.modules.ctx().guard;
+      const authenticator = app?.module.auth.authenticator;
+
+      let skipped = shouldSkip(c, options?.skip) || !app?.module.auth.enabled;
+
+      // make sure to only register once
+      if (c.get("auth_registered")) {
+         skipped = true;
+         console.warn(`auth middleware already registered for ${getPath(c)}`);
+      } else {
+         c.set("auth_registered", true);
+
+         if (!skipped) {
+            const resolved = c.get("auth_resolved");
+            if (!resolved) {
+               if (!app?.module.auth.enabled) {
+                  guard?.setUserContext(undefined);
+               } else {
+                  guard?.setUserContext(await authenticator?.resolveAuthFromRequest(c));
+                  c.set("auth_resolved", true);
+               }
+            }
+         }
+      }
+
+      await next();
+
+      if (!skipped) {
+         // renew cookie if applicable
+         authenticator?.requestCookieRefresh(c);
+      }
+
+      // release
+      guard?.setUserContext(undefined);
+      authenticator?.resetUser();
+      c.set("auth_resolved", false);
+   });
+
+export const permission = (
+   permission: Permission | Permission[],
+   options?: {
+      onGranted?: (c: Context<ServerEnv>) => Promise<Response | void | undefined>;
+      onDenied?: (c: Context<ServerEnv>) => Promise<Response | void | undefined>;
+   }
+) =>
+   // @ts-ignore
+   createMiddleware<ServerEnv>(async (c, next) => {
+      const app = c.get("app");
+      //console.log("skip?", c.get("auth_skip"));
+
+      // in tests, app is not defined
+      if (!c.get("auth_registered") || !app) {
+         const msg = `auth middleware not registered, cannot check permissions for ${getPath(c)}`;
+         if (app?.module.auth.enabled) {
+            throw new Error(msg);
+         } else {
+            console.warn(msg);
+         }
+      } else if (!c.get("auth_skip")) {
+         const guard = app.modules.ctx().guard;
+         const permissions = Array.isArray(permission) ? permission : [permission];
+
+         if (options?.onGranted || options?.onDenied) {
+            let returned: undefined | void | Response;
+            if (permissions.every((p) => guard.granted(p))) {
+               returned = await options?.onGranted?.(c);
+            } else {
+               returned = await options?.onDenied?.(c);
+            }
+            if (returned instanceof Response) {
+               return returned;
+            }
+         } else {
+            permissions.some((p) => guard.throwUnlessGranted(p));
+         }
+      }
+
+      await next();
+   });

app/src/cli/commands/run/platform.ts

@@ -1,5 +1,6 @@
 import path from "node:path";
 import type { Config } from "@libsql/client/node";
+import { config } from "core";
 import type { MiddlewareHandler } from "hono";
 import open from "open";
 import { fileExists, getRelativeDistPath } from "../../utils/sys";
@@ -26,7 +27,7 @@ export async function serveStatic(server: Platform): Promise<MiddlewareHandler>
 }
 
 export async function attachServeStatic(app: any, platform: Platform) {
-   app.module.server.client.get("/*", await serveStatic(platform));
+   app.module.server.client.get(config.server.assets_path + "*", await serveStatic(platform));
 }
 
 export async function startServer(server: Platform, app: any, options: { port: number }) {

app/src/cli/commands/user.ts

@@ -35,9 +35,11 @@ async function action(action: "create" | "update", options: any) {
 }
 
 async function create(app: App, options: any) {
-   const config = app.module.auth.toJSON(true);
    const strategy = app.module.auth.authenticator.strategy("password") as PasswordStrategy;
-   const users_entity = config.entity_name as "users";
+
+   if (!strategy) {
+      throw new Error("Password strategy not configured");
+   }
 
    const email = await $text({
       message: "Enter email",
@@ -65,16 +67,11 @@ async function create(app: App, options: any) {
    }
 
    try {
-      const mutator = app.modules.ctx().em.mutator(users_entity);
-      mutator.__unstable_toggleSystemEntityCreation(false);
-      const res = await mutator.insertOne({
+      const created = await app.createUser({
          email,
-         strategy: "password",
-         strategy_value: await strategy.hash(password as string)
-      });
-      mutator.__unstable_toggleSystemEntityCreation(true);
-
-      console.log("Created:", res.data);
+         password: await strategy.hash(password as string)
+      })
+      console.log("Created:", created);
    } catch (e) {
       console.error("Error", e);
    }
@@ -141,4 +138,4 @@ async function update(app: App, options: any) {
    } catch (e) {
       console.error("Error", e);
    }
-}
+}

app/src/core/config.ts

@@ -5,12 +5,19 @@ import type { Generated } from "kysely";
 
 export type PrimaryFieldType = number | Generated<number>;
 
-// biome-ignore lint/suspicious/noEmptyInterface: <explanation>
-export interface DB {}
+export interface DB {
+   // make sure to make unknown as "any"
+   [key: string]: {
+      id: PrimaryFieldType;
+      [key: string]: any;
+   };
+}
 
 export const config = {
    server: {
-      default_port: 1337
+      default_port: 1337,
+      // resetted to root for now, bc bundling with vite
+      assets_path: "/"
    },
    data: {
       default_primary_field: "id"

app/src/core/errors.ts

@@ -1,6 +1,7 @@
 export class Exception extends Error {
    code = 400;
    override name = "Exception";
+   protected _context = undefined;
 
    constructor(message: string, code?: number) {
       super(message);
@@ -9,11 +10,16 @@ export class Exception extends Error {
       }
    }
 
+   context(context: any) {
+      this._context = context;
+      return this;
+   }
+
    toJSON() {
       return {
          error: this.message,
-         type: this.name
-         //message: this.message
+         type: this.name,
+         context: this._context
       };
    }
 }

app/src/core/events/Event.ts

@@ -1,17 +1,38 @@
-export abstract class Event<Params = any> {
+export type EventClass = {
+   new (params: any): Event<any, any>;
+   slug: string;
+};
+
+export abstract class Event<Params = any, Returning = void> {
+   _returning!: Returning;
+
    /**
     * Unique event slug
     * Must be static, because registering events is done by class
     */
    static slug: string = "untitled-event";
    params: Params;
+   returned: boolean = false;
+
+   validate(value: Returning): Event<Params, Returning> | void {
+      throw new EventReturnedWithoutValidation(this as any, value);
+   }
+
+   protected clone<This extends Event<Params, Returning> = Event<Params, Returning>>(
+      this: This,
+      params: Params
+   ): This {
+      const cloned = new (this.constructor as any)(params);
+      cloned.returned = true;
+      return cloned as This;
+   }
 
    constructor(params: Params) {
       this.params = params;
    }
 }
 
-// @todo: current workaround: potentially there is none and that's the way
+// @todo: current workaround: potentially there is "none" and that's the way
 export class NoParamEvent extends Event<null> {
    static override slug: string = "noparam-event";
 
@@ -19,3 +40,19 @@ export class NoParamEvent extends Event<null> {
       super(null);
    }
 }
+
+export class InvalidEventReturn extends Error {
+   constructor(expected: string, given: string) {
+      super(`Expected "${expected}", got "${given}"`);
+   }
+}
+
+export class EventReturnedWithoutValidation extends Error {
+   constructor(
+      event: EventClass,
+      public data: any
+   ) {
+      // @ts-expect-error slug is static
+      super(`Event "${event.constructor.slug}" returned without validation`);
+   }
+}

app/src/core/events/EventListener.ts

@@ -4,15 +4,16 @@ import type { EventClass } from "./EventManager";
 export const ListenerModes = ["sync", "async"] as const;
 export type ListenerMode = (typeof ListenerModes)[number];
 
-export type ListenerHandler<E extends Event = Event> = (
+export type ListenerHandler<E extends Event<any, any>> = (
    event: E,
-   slug: string,
-) => Promise<void> | void;
+   slug: string
+) => E extends Event<any, infer R> ? R | Promise<R | void> : never;
 
 export class EventListener<E extends Event = Event> {
    mode: ListenerMode = "async";
    event: EventClass;
    handler: ListenerHandler<E>;
+   once: boolean = false;
 
    constructor(event: EventClass, handler: ListenerHandler<E>, mode: ListenerMode = "async") {
       this.event = event;

app/src/core/events/EventManager.ts

@@ -1,14 +1,19 @@
-import type { Event } from "./Event";
+import { type Event, type EventClass, InvalidEventReturn } from "./Event";
 import { EventListener, type ListenerHandler, type ListenerMode } from "./EventListener";
 
+export type RegisterListenerConfig =
+   | ListenerMode
+   | {
+        mode?: ListenerMode;
+        once?: boolean;
+     };
+
 export interface EmitsEvents {
    emgr: EventManager;
 }
 
-export type EventClass = {
-   new (params: any): Event;
-   slug: string;
-};
+// for compatibility, moved it to Event.ts
+export type { EventClass };
 
 export class EventManager<
    RegisteredEvents extends Record<string, EventClass> = Record<string, EventClass>
@@ -17,16 +22,20 @@ export class EventManager<
    protected listeners: EventListener[] = [];
    enabled: boolean = true;
 
-   constructor(events?: RegisteredEvents, listeners?: EventListener[]) {
+   constructor(
+      events?: RegisteredEvents,
+      private options?: {
+         listeners?: EventListener[];
+         onError?: (event: Event, e: unknown) => void;
+         onInvalidReturn?: (event: Event, e: InvalidEventReturn) => void;
+         asyncExecutor?: typeof Promise.all;
+      }
+   ) {
       if (events) {
          this.registerEvents(events);
       }
 
-      if (listeners) {
-         for (const listener of listeners) {
-            this.addListener(listener);
-         }
-      }
+      options?.listeners?.forEach((l) => this.addListener(l));
    }
 
    enable() {
@@ -82,9 +91,11 @@ export class EventManager<
       return !!this.events.find((e) => slug === e.slug);
    }
 
-   protected throwIfEventNotRegistered(event: EventClass) {
-      if (!this.eventExists(event)) {
-         throw new Error(`Event "${event.slug}" not registered`);
+   protected throwIfEventNotRegistered(event: EventClass | Event | string) {
+      if (!this.eventExists(event as any)) {
+         // @ts-expect-error
+         const name = event.constructor?.slug ?? event.slug ?? event;
+         throw new Error(`Event "${name}" not registered`);
       }
    }
 
@@ -117,55 +128,108 @@ export class EventManager<
       return this;
    }
 
+   protected createEventListener(
+      _event: EventClass | string,
+      handler: ListenerHandler<any>,
+      _config: RegisterListenerConfig = "async"
+   ) {
+      const event =
+         typeof _event === "string" ? this.events.find((e) => e.slug === _event)! : _event;
+      const config = typeof _config === "string" ? { mode: _config } : _config;
+      const listener = new EventListener(event, handler, config.mode);
+      if (config.once) {
+         listener.once = true;
+      }
+      this.addListener(listener as any);
+   }
+
    onEvent<ActualEvent extends EventClass, Instance extends InstanceType<ActualEvent>>(
       event: ActualEvent,
       handler: ListenerHandler<Instance>,
-      mode: ListenerMode = "async"
+      config?: RegisterListenerConfig
    ) {
-      this.throwIfEventNotRegistered(event);
-
-      const listener = new EventListener(event, handler, mode);
-      this.addListener(listener as any);
+      this.createEventListener(event, handler, config);
    }
 
    on<Params = any>(
       slug: string,
       handler: ListenerHandler<Event<Params>>,
-      mode: ListenerMode = "async"
+      config?: RegisterListenerConfig
    ) {
-      const event = this.events.find((e) => e.slug === slug);
-      if (!event) {
-         throw new Error(`Event "${slug}" not registered`);
-      }
-
-      this.onEvent(event, handler, mode);
+      this.createEventListener(slug, handler, config);
    }
 
-   onAny(handler: ListenerHandler<Event<unknown>>, mode: ListenerMode = "async") {
-      this.events.forEach((event) => this.onEvent(event, handler, mode));
+   onAny(handler: ListenerHandler<Event<unknown>>, config?: RegisterListenerConfig) {
+      this.events.forEach((event) => this.onEvent(event, handler, config));
    }
 
-   async emit(event: Event) {
+   protected executeAsyncs(promises: (() => Promise<void>)[]) {
+      const executor = this.options?.asyncExecutor ?? ((e) => Promise.all(e));
+      executor(promises.map((p) => p())).then(() => void 0);
+   }
+
+   async emit<Actual extends Event<any, any>>(event: Actual): Promise<Actual> {
       // @ts-expect-error slug is static
       const slug = event.constructor.slug;
       if (!this.enabled) {
          console.log("EventManager disabled, not emitting", slug);
-         return;
+         return event;
       }
 
       if (!this.eventExists(event)) {
          throw new Error(`Event "${slug}" not registered`);
       }
 
-      const listeners = this.listeners.filter((listener) => listener.event.slug === slug);
-      //console.log("---!-- emitting", slug, listeners.length);
+      const syncs: EventListener[] = [];
+      const asyncs: (() => Promise<void>)[] = [];
+
+      this.listeners = this.listeners.filter((listener) => {
+         // if no match, keep and ignore
+         if (listener.event.slug !== slug) return true;
 
-      for (const listener of listeners) {
          if (listener.mode === "sync") {
-            await listener.handler(event, listener.event.slug);
+            syncs.push(listener);
          } else {
-            listener.handler(event, listener.event.slug);
+            asyncs.push(async () => await listener.handler(event, listener.event.slug));
+         }
+         // Remove if `once` is true, otherwise keep
+         return !listener.once;
+      });
+
+      // execute asyncs
+      this.executeAsyncs(asyncs);
+
+      // execute syncs
+      let _event: Actual = event;
+      for (const listener of syncs) {
+         try {
+            const return_value = (await listener.handler(_event, listener.event.slug)) as any;
+
+            if (typeof return_value !== "undefined") {
+               const newEvent = _event.validate(return_value);
+               // @ts-expect-error slug is static
+               if (newEvent && newEvent.constructor.slug === slug) {
+                  if (!newEvent.returned) {
+                     throw new Error(
+                        // @ts-expect-error slug is static
+                        `Returned event ${newEvent.constructor.slug} must be marked as returned.`
+                     );
+                  }
+                  _event = newEvent as Actual;
+               }
+            }
+         } catch (e) {
+            if (e instanceof InvalidEventReturn) {
+               this.options?.onInvalidReturn?.(_event, e);
+               console.warn(`Invalid return of event listener for "${slug}": ${e.message}`);
+            } else if (this.options?.onError) {
+               this.options.onError(_event, e);
+            } else {
+               throw e;
+            }
          }
       }
+
+      return _event;
    }
 }

app/src/core/events/index.ts

@@ -1,8 +1,8 @@
-export { Event, NoParamEvent } from "./Event";
+export { Event, NoParamEvent, InvalidEventReturn } from "./Event";
 export {
    EventListener,
    ListenerModes,
    type ListenerMode,
-   type ListenerHandler,
+   type ListenerHandler
 } from "./EventListener";
 export { EventManager, type EmitsEvents, type EventClass } from "./EventManager";

app/src/core/object/SchemaObject.ts

@@ -1,10 +1,10 @@
-import { cloneDeep, get, has, mergeWith, omit, set } from "lodash-es";
+import { get, has, omit, set } from "lodash-es";
 import {
    Default,
    type Static,
    type TObject,
    getFullPathKeys,
-   mark,
+   mergeObjectWith,
    parse,
    stripMark
 } from "../utils";
@@ -33,7 +33,7 @@ export class SchemaObject<Schema extends TObject> {
    ) {
       this._default = Default(_schema, {} as any) as any;
       this._value = initial
-         ? parse(_schema, cloneDeep(initial as any), {
+         ? parse(_schema, structuredClone(initial as any), {
               forceParse: this.isForceParse(),
               skipMark: this.isForceParse()
            })
@@ -64,8 +64,12 @@ export class SchemaObject<Schema extends TObject> {
       return this._config;
    }
 
+   clone() {
+      return structuredClone(this._config);
+   }
+
    async set(config: Static<Schema>, noEmit?: boolean): Promise<Static<Schema>> {
-      const valid = parse(this._schema, cloneDeep(config) as any, {
+      const valid = parse(this._schema, structuredClone(config) as any, {
          forceParse: true,
          skipMark: this.isForceParse()
       });
@@ -114,7 +118,7 @@ export class SchemaObject<Schema extends TObject> {
    }
 
    async patch(path: string, value: any): Promise<[Partial<Static<Schema>>, Static<Schema>]> {
-      const current = cloneDeep(this._config);
+      const current = this.clone();
       const partial = path.length > 0 ? (set({}, path, value) as Partial<Static<Schema>>) : value;
 
       this.throwIfRestricted(partial);
@@ -122,15 +126,20 @@ export class SchemaObject<Schema extends TObject> {
 
       // overwrite arrays and primitives, only deep merge objects
       // @ts-ignore
-      const config = mergeWith(current, partial, (objValue, srcValue) => {
+      //console.log("---alt:new", _jsonp(mergeObject(current, partial)));
+      const config = mergeObjectWith(current, partial, (objValue, srcValue) => {
          if (Array.isArray(objValue) && Array.isArray(srcValue)) {
             return srcValue;
          }
       });
+      //console.log("---new", _jsonp(config));
 
       //console.log("overwritePaths", this.options?.overwritePaths);
       if (this.options?.overwritePaths) {
-         const keys = getFullPathKeys(value).map((k) => path + "." + k);
+         const keys = getFullPathKeys(value).map((k) => {
+            // only prepend path if given
+            return path.length > 0 ? path + "." + k : k;
+         });
          const overwritePaths = keys.filter((k) => {
             return this.options?.overwritePaths?.some((p) => {
                if (typeof p === "string") {
@@ -161,14 +170,14 @@ export class SchemaObject<Schema extends TObject> {
          }
       }
 
-      //console.log("patch", { path, value, partial, config, current });
+      //console.log("patch", _jsonp({ path, value, partial, config, current }));
 
       const newConfig = await this.set(config);
       return [partial, newConfig];
    }
 
    async overwrite(path: string, value: any): Promise<[Partial<Static<Schema>>, Static<Schema>]> {
-      const current = cloneDeep(this._config);
+      const current = this.clone();
       const partial = path.length > 0 ? (set({}, path, value) as Partial<Static<Schema>>) : value;
 
       this.throwIfRestricted(partial);
@@ -189,7 +198,7 @@ export class SchemaObject<Schema extends TObject> {
       if (p.length > 1) {
          const parent = p.slice(0, -1).join(".");
          if (!has(this._config, parent)) {
-            console.log("parent", parent, JSON.stringify(this._config, null, 2));
+            //console.log("parent", parent, JSON.stringify(this._config, null, 2));
             throw new Error(`Parent path "${parent}" does not exist`);
          }
       }
@@ -204,7 +213,7 @@ export class SchemaObject<Schema extends TObject> {
          throw new Error(`Path "${path}" does not exist`);
       }
 
-      const current = cloneDeep(this._config);
+      const current = this.clone();
       const removed = get(current, path) as Partial<Static<Schema>>;
       const config = omit(current, path);
       const newConfig = await this.set(config);

app/src/core/object/query/query.ts

@@ -49,7 +49,7 @@ type LiteralExpressionCondition<Exps extends Expressions> = {
    [key: string]: Primitive | ExpressionCondition<Exps>;
 };
 
-const OperandOr = "$or";
+const OperandOr = "$or" as const;
 type OperandCondition<Exps extends Expressions> = {
    [OperandOr]?: LiteralExpressionCondition<Exps> | ExpressionCondition<Exps>;
 };

app/src/core/server/flash.ts

@@ -4,14 +4,12 @@ import { setCookie } from "hono/cookie";
 const flash_key = "__bknd_flash";
 export type FlashMessageType = "error" | "warning" | "success" | "info";
 
-export async function addFlashMessage(
-   c: Context,
-   message: string,
-   type: FlashMessageType = "info"
-) {
-   setCookie(c, flash_key, JSON.stringify({ type, message }), {
-      path: "/"
-   });
+export function addFlashMessage(c: Context, message: string, type: FlashMessageType = "info") {
+   if (c.req.header("Accept")?.includes("text/html")) {
+      setCookie(c, flash_key, JSON.stringify({ type, message }), {
+         path: "/"
+      });
+   }
 }
 
 function getCookieValue(name) {

app/src/core/utils/index.ts

@@ -11,3 +11,5 @@ export * from "./crypto";
 export * from "./uuid";
 export { FromSchema } from "./typebox/from-schema";
 export * from "./test";
+export * from "./runtime";
+export * from "./numbers";

app/src/core/utils/numbers.ts

@@ -0,0 +1,5 @@
+export function clampNumber(value: number, min: number, max: number): number {
+   const lower = Math.min(min, max);
+   const upper = Math.max(min, max);
+   return Math.max(lower, Math.min(value, upper));
+}

app/src/core/utils/objects.ts

@@ -4,6 +4,14 @@ export function _jsonp(obj: any, space = 2): string {
    return JSON.stringify(obj, null, space);
 }
 
+export function isPlainObject(value: unknown): value is Record<string, unknown> {
+   return Object.prototype.toString.call(value) === "[object Object]";
+}
+
+export function isObject(value: unknown): value is Record<string, unknown> {
+   return value !== null && typeof value === "object";
+}
+
 export function safelyParseObjectValues<T extends { [key: string]: any }>(obj: T): T {
    return Object.entries(obj).reduce((acc, [key, value]) => {
       try {
@@ -97,15 +105,6 @@ export function objectEach<T extends Record<string, any>, U>(
    );
 }
 
-/**
- * Simple object check.
- * @param item
- * @returns {boolean}
- */
-export function isObject(item) {
-   return item && typeof item === "object" && !Array.isArray(item);
-}
-
 /**
  * Deep merge two objects.
  * @param target
@@ -197,3 +196,73 @@ export function objectCleanEmpty<Obj extends { [key: string]: any }>(obj: Obj):
       return acc;
    }, {} as any);
 }
+
+/**
+ * Lodash's merge implementation caused issues in Next.js environments
+ * From: https://thescottyjam.github.io/snap.js/#!/nolodash/merge
+ * NOTE: This mutates `object`. It also may mutate anything that gets attached to `object` during the merge.
+ * @param object
+ * @param sources
+ */
+export function mergeObject(object, ...sources) {
+   for (const source of sources) {
+      for (const [key, value] of Object.entries(source)) {
+         if (value === undefined) {
+            continue;
+         }
+
+         // These checks are a week attempt at mimicking the various edge-case behaviors
+         // that Lodash's `_.merge()` exhibits. Feel free to simplify and
+         // remove checks that you don't need.
+         if (!isPlainObject(value) && !Array.isArray(value)) {
+            object[key] = value;
+         } else if (Array.isArray(value) && !Array.isArray(object[key])) {
+            object[key] = value;
+         } else if (!isObject(object[key])) {
+            object[key] = value;
+         } else {
+            mergeObject(object[key], value);
+         }
+      }
+   }
+
+   return object;
+}
+
+/**
+ * Lodash's mergeWith implementation caused issues in Next.js environments
+ * From: https://thescottyjam.github.io/snap.js/#!/nolodash/mergeWith
+ * NOTE: This mutates `object`. It also may mutate anything that gets attached to `object` during the merge.
+ * @param object
+ * @param sources
+ * @param customizer
+ */
+export function mergeObjectWith(object, source, customizer) {
+   for (const [key, value] of Object.entries(source)) {
+      const mergedValue = customizer(object[key], value, key, object, source);
+      if (mergedValue !== undefined) {
+         object[key] = mergedValue;
+         continue;
+      }
+      // Otherwise, fall back to default behavior
+
+      if (value === undefined) {
+         continue;
+      }
+
+      // These checks are a week attempt at mimicking the various edge-case behaviors
+      // that Lodash's `_.merge()` exhibits. Feel free to simplify and
+      // remove checks that you don't need.
+      if (!isPlainObject(value) && !Array.isArray(value)) {
+         object[key] = value;
+      } else if (Array.isArray(value) && !Array.isArray(object[key])) {
+         object[key] = value;
+      } else if (!isObject(object[key])) {
+         object[key] = value;
+      } else {
+         mergeObjectWith(object[key], value, customizer);
+      }
+   }
+
+   return object;
+}

app/src/core/utils/runtime.ts

@@ -0,0 +1,41 @@
+import { getRuntimeKey as honoGetRuntimeKey } from "hono/adapter";
+
+/**
+ * Adds additional checks for nextjs
+ */
+export function getRuntimeKey(): string {
+   const global = globalThis as any;
+
+   // Detect Next.js server-side runtime
+   if (global?.process?.env?.NEXT_RUNTIME === "nodejs") {
+      return "nextjs";
+   }
+
+   // Detect Next.js edge runtime
+   if (global?.process?.env?.NEXT_RUNTIME === "edge") {
+      return "nextjs-edge";
+   }
+
+   // Detect Next.js client-side runtime
+   // @ts-ignore
+   if (typeof window !== "undefined" && window.__NEXT_DATA__) {
+      return "nextjs-client";
+   }
+
+   return honoGetRuntimeKey();
+}
+
+const features = {
+   // supports the redirect of not full qualified addresses
+   // not supported in nextjs
+   redirects_non_fq: true
+};
+
+export function runtimeSupports(feature: keyof typeof features) {
+   const runtime = getRuntimeKey();
+   if (runtime.startsWith("nextjs")) {
+      features.redirects_non_fq = false;
+   }
+
+   return features[feature];
+}

app/src/core/utils/strings.ts

@@ -104,3 +104,14 @@ export function replaceSimplePlaceholders(str: string, vars: Record<string, any>
       return key in vars ? vars[key] : match;
    });
 }
+
+export function patternMatch(target: string, pattern: RegExp | string): boolean {
+   if (pattern instanceof RegExp) {
+      return pattern.test(target);
+   } else if (typeof pattern === "string" && pattern.startsWith("/")) {
+      return new RegExp(pattern).test(target);
+   } else if (typeof pattern === "string") {
+      return target.startsWith(pattern);
+   }
+   return false;
+}

app/src/core/utils/test.ts

@@ -7,7 +7,7 @@ const _oldConsoles = {
 
 export async function withDisabledConsole<R>(
    fn: () => Promise<R>,
-   severities: ConsoleSeverity[] = ["log"]
+   severities: ConsoleSeverity[] = ["log", "warn", "error"]
 ): Promise<R> {
    const _oldConsoles = {
       log: console.log,
@@ -30,7 +30,7 @@ export async function withDisabledConsole<R>(
    }
 }
 
-export function disableConsoleLog(severities: ConsoleSeverity[] = ["log"]) {
+export function disableConsoleLog(severities: ConsoleSeverity[] = ["log", "warn"]) {
    severities.forEach((severity) => {
       console[severity] = () => null;
    });

app/src/core/utils/typebox/index.ts

@@ -115,6 +115,7 @@ export function parse<Schema extends TSchema = TSchema>(
    } else if (options?.onError) {
       options.onError(Errors(schema, data));
    } else {
+      //console.warn("errors", JSON.stringify([...Errors(schema, data)], null, 2));
       throw new TypeInvalidError(schema, data);
    }
 

app/src/data/AppData.ts

@@ -13,7 +13,15 @@ import { type AppDataConfig, dataConfigSchema } from "./data-schema";
 
 export class AppData extends Module<typeof dataConfigSchema> {
    override async build() {
-      const entities = transformObject(this.config.entities ?? {}, (entityConfig, name) => {
+      const {
+         entities: _entities = {},
+         relations: _relations = {},
+         indices: _indices = {}
+      } = this.config;
+
+      this.ctx.logger.context("AppData").log("building with entities", Object.keys(_entities));
+
+      const entities = transformObject(_entities, (entityConfig, name) => {
          return constructEntity(name, entityConfig);
       });
 
@@ -21,14 +29,14 @@ export class AppData extends Module<typeof dataConfigSchema> {
          const name = typeof _e === "string" ? _e : _e.name;
          const entity = entities[name];
          if (entity) return entity;
-         throw new Error(`Entity "${name}" not found`);
+         throw new Error(`[AppData] Entity "${name}" not found`);
       };
 
-      const relations = transformObject(this.config.relations ?? {}, (relation) =>
+      const relations = transformObject(_relations, (relation) =>
          constructRelation(relation, _entity)
       );
 
-      const indices = transformObject(this.config.indices ?? {}, (index, name) => {
+      const indices = transformObject(_indices, (index, name) => {
          const entity = _entity(index.entity)!;
          const fields = index.fields.map((f) => entity.field(f)!);
          return new EntityIndex(entity, fields, index.unique, name);
@@ -52,6 +60,7 @@ export class AppData extends Module<typeof dataConfigSchema> {
       );
       this.ctx.guard.registerPermissions(Object.values(DataPermissions));
 
+      this.ctx.logger.clear();
       this.setBuilt();
    }
 
@@ -69,18 +78,9 @@ export class AppData extends Module<typeof dataConfigSchema> {
    }
 
    override getOverwritePaths() {
-      return [
-         /^entities\..*\.config$/,
-         /^entities\..*\.fields\..*\.config$/
-         ///^entities\..*\.fields\..*\.config\.schema$/
-      ];
+      return [/^entities\..*\.config$/, /^entities\..*\.fields\..*\.config$/];
    }
 
-   /*registerController(server: AppServer) {
-      console.log("adding data controller to", this.basepath);
-      server.add(this.basepath, new DataController(this.em));
-   }*/
-
    override toJSON(secrets?: boolean): AppDataConfig {
       return {
          ...this.config,

app/src/data/api/DataApi.ts

@@ -1,15 +1,17 @@
 import type { DB } from "core";
-import type { EntityData, RepoQuery, RepositoryResponse } from "data";
+import type { EntityData, RepoQuery, RepoQueryIn, RepositoryResponse } from "data";
 import { type BaseModuleApiOptions, ModuleApi, type PrimaryFieldType } from "modules";
 
 export type DataApiOptions = BaseModuleApiOptions & {
-   defaultQuery?: Partial<RepoQuery>;
+   queryLengthLimit: number;
+   defaultQuery: Partial<RepoQuery>;
 };
 
 export class DataApi extends ModuleApi<DataApiOptions> {
    protected override getDefaultOptions(): Partial<DataApiOptions> {
       return {
          basepath: "/api/data",
+         queryLengthLimit: 1000,
          defaultQuery: {
             limit: 10
          }
@@ -19,26 +21,32 @@ export class DataApi extends ModuleApi<DataApiOptions> {
    readOne<E extends keyof DB | string, Data = E extends keyof DB ? DB[E] : EntityData>(
       entity: E,
       id: PrimaryFieldType,
-      query: Partial<Omit<RepoQuery, "where" | "limit" | "offset">> = {}
+      query: Omit<RepoQueryIn, "where" | "limit" | "offset"> = {}
    ) {
       return this.get<Pick<RepositoryResponse<Data>, "meta" | "data">>([entity as any, id], query);
    }
 
    readMany<E extends keyof DB | string, Data = E extends keyof DB ? DB[E] : EntityData>(
       entity: E,
-      query: Partial<RepoQuery> = {}
+      query: RepoQueryIn = {}
    ) {
-      return this.get<Pick<RepositoryResponse<Data[]>, "meta" | "data">>(
-         [entity as any],
-         query ?? this.options.defaultQuery
-      );
+      type T = Pick<RepositoryResponse<Data[]>, "meta" | "data">;
+
+      const input = query ?? this.options.defaultQuery;
+      const req = this.get<T>([entity as any], input);
+
+      if (req.request.url.length <= this.options.queryLengthLimit) {
+         return req;
+      }
+
+      return this.post<T>([entity as any, "query"], input);
    }
 
    readManyByReference<
       E extends keyof DB | string,
       R extends keyof DB | string,
       Data = R extends keyof DB ? DB[R] : EntityData
-   >(entity: E, id: PrimaryFieldType, reference: R, query: Partial<RepoQuery> = {}) {
+   >(entity: E, id: PrimaryFieldType, reference: R, query: RepoQueryIn = {}) {
       return this.get<Pick<RepositoryResponse<Data[]>, "meta" | "data">>(
          [entity as any, id, reference],
          query ?? this.options.defaultQuery

app/src/data/api/DataController.ts

@@ -1,32 +1,26 @@
-import { type ClassController, isDebug, tbValidator as tb } from "core";
-import { StringEnum, Type, objectCleanEmpty, objectTransform } from "core/utils";
+import { isDebug, tbValidator as tb } from "core";
+import { StringEnum, Type } from "core/utils";
 import {
    DataPermissions,
    type EntityData,
    type EntityManager,
-   FieldClassMap,
    type MutatorResponse,
-   PrimaryField,
    type RepoQuery,
    type RepositoryResponse,
-   TextField,
    querySchema
 } from "data";
-import { Hono } from "hono";
 import type { Handler } from "hono/types";
 import type { ModuleBuildContext } from "modules";
+import { Controller } from "modules/Controller";
 import * as SystemPermissions from "modules/permissions";
-import { type AppDataConfig, FIELDS } from "../data-schema";
+import type { AppDataConfig } from "../data-schema";
 
-export class DataController implements ClassController {
+export class DataController extends Controller {
    constructor(
       private readonly ctx: ModuleBuildContext,
       private readonly config: AppDataConfig
    ) {
-      /*console.log(
-         "data controller",
-         this.em.entities.map((e) => e.name)
-      );*/
+      super();
    }
 
    get em(): EntityManager<any> {
@@ -74,8 +68,10 @@ export class DataController implements ClassController {
       }
    }
 
-   getController(): Hono<any> {
-      const hono = new Hono();
+   override getController() {
+      const { permission, auth } = this.middlewares;
+      const hono = this.create().use(auth(), permission(SystemPermissions.accessApi));
+
       const definedEntities = this.em.entities.map((e) => e.name);
       const tbNumber = Type.Transform(Type.String({ pattern: "^[1-9][0-9]{0,}$" }))
          .Decode(Number.parseInt)
@@ -89,11 +85,6 @@ export class DataController implements ClassController {
          return func;
       }
 
-      hono.use("*", async (c, next) => {
-         this.ctx.guard.throwUnlessGranted(SystemPermissions.accessApi);
-         await next();
-      });
-
       // info
       hono.get(
          "/",
@@ -104,9 +95,7 @@ export class DataController implements ClassController {
       );
 
       // sync endpoint
-      hono.get("/sync", async (c) => {
-         this.guard.throwUnlessGranted(DataPermissions.databaseSync);
-
+      hono.get("/sync", permission(DataPermissions.databaseSync), async (c) => {
          const force = c.req.query("force") === "1";
          const drop = c.req.query("drop") === "1";
          //console.log("force", force);
@@ -126,10 +115,9 @@ export class DataController implements ClassController {
          // fn: count
          .post(
             "/:entity/fn/count",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity } = c.req.valid("param");
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -143,10 +131,9 @@ export class DataController implements ClassController {
          // fn: exists
          .post(
             "/:entity/fn/exists",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity } = c.req.valid("param");
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -163,8 +150,7 @@ export class DataController implements ClassController {
        */
       hono
          // read entity schema
-         .get("/schema.json", async (c) => {
-            this.guard.throwUnlessGranted(DataPermissions.entityRead);
+         .get("/schema.json", permission(DataPermissions.entityRead), async (c) => {
             const $id = `${this.config.basepath}/schema.json`;
             const schemas = Object.fromEntries(
                this.em.entities.map((e) => [
@@ -183,6 +169,7 @@ export class DataController implements ClassController {
          // read schema
          .get(
             "/schemas/:entity/:context?",
+            permission(DataPermissions.entityRead),
             tb(
                "param",
                Type.Object({
@@ -191,8 +178,6 @@ export class DataController implements ClassController {
                })
             ),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                //console.log("request", c.req.raw);
                const { entity, context } = c.req.param();
                if (!this.entityExists(entity)) {
@@ -216,11 +201,10 @@ export class DataController implements ClassController {
          // read many
          .get(
             "/:entity",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             tb("query", querySchema),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                //console.log("request", c.req.raw);
                const { entity } = c.req.param();
                if (!this.entityExists(entity)) {
@@ -238,6 +222,7 @@ export class DataController implements ClassController {
          // read one
          .get(
             "/:entity/:id",
+            permission(DataPermissions.entityRead),
             tb(
                "param",
                Type.Object({
@@ -246,11 +231,7 @@ export class DataController implements ClassController {
                })
             ),
             tb("query", querySchema),
-            /*zValidator("param", z.object({ entity: z.string(), id: z.coerce.number() })),
-            zValidator("query", repoQuerySchema),*/
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity, id } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -264,6 +245,7 @@ export class DataController implements ClassController {
          // read many by reference
          .get(
             "/:entity/:id/:reference",
+            permission(DataPermissions.entityRead),
             tb(
                "param",
                Type.Object({
@@ -274,8 +256,6 @@ export class DataController implements ClassController {
             ),
             tb("query", querySchema),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity, id, reference } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -292,17 +272,16 @@ export class DataController implements ClassController {
          // func query
          .post(
             "/:entity/query",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             tb("json", querySchema),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
                }
                const options = (await c.req.valid("json")) as RepoQuery;
-               console.log("options", options);
+               //console.log("options", options);
                const result = await this.em.repository(entity).findMany(options);
 
                return c.json(this.repoResult(result), { status: result.data ? 200 : 404 });
@@ -314,25 +293,27 @@ export class DataController implements ClassController {
        */
       // insert one
       hono
-         .post("/:entity", tb("param", Type.Object({ entity: Type.String() })), async (c) => {
-            this.guard.throwUnlessGranted(DataPermissions.entityCreate);
+         .post(
+            "/:entity",
+            permission(DataPermissions.entityCreate),
+            tb("param", Type.Object({ entity: Type.String() })),
+            async (c) => {
+               const { entity } = c.req.param();
+               if (!this.entityExists(entity)) {
+                  return c.notFound();
+               }
+               const body = (await c.req.json()) as EntityData;
+               const result = await this.em.mutator(entity).insertOne(body);
 
-            const { entity } = c.req.param();
-            if (!this.entityExists(entity)) {
-               return c.notFound();
+               return c.json(this.mutatorResult(result), 201);
             }
-            const body = (await c.req.json()) as EntityData;
-            const result = await this.em.mutator(entity).insertOne(body);
-
-            return c.json(this.mutatorResult(result), 201);
-         })
+         )
          // update one
          .patch(
             "/:entity/:id",
+            permission(DataPermissions.entityUpdate),
             tb("param", Type.Object({ entity: Type.String(), id: tbNumber })),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityUpdate);
-
                const { entity, id } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -346,6 +327,8 @@ export class DataController implements ClassController {
          // delete one
          .delete(
             "/:entity/:id",
+
+            permission(DataPermissions.entityDelete),
             tb("param", Type.Object({ entity: Type.String(), id: tbNumber })),
             async (c) => {
                this.guard.throwUnlessGranted(DataPermissions.entityDelete);
@@ -363,11 +346,10 @@ export class DataController implements ClassController {
          // delete many
          .delete(
             "/:entity",
+            permission(DataPermissions.entityDelete),
             tb("param", Type.Object({ entity: Type.String() })),
             tb("json", querySchema.properties.where),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityDelete);
-
                //console.log("request", c.req.raw);
                const { entity } = c.req.param();
                if (!this.entityExists(entity)) {

app/src/data/connection/SqliteLocalConnection.ts

@@ -1,6 +1,5 @@
-import type { DatabaseIntrospector, SqliteDatabase } from "kysely";
+import { type DatabaseIntrospector, ParseJSONResultsPlugin, type SqliteDatabase } from "kysely";
 import { Kysely, SqliteDialect } from "kysely";
-import { DeserializeJsonValuesPlugin } from "../plugins/DeserializeJsonValuesPlugin";
 import { SqliteConnection } from "./SqliteConnection";
 import { SqliteIntrospector } from "./SqliteIntrospector";
 
@@ -14,7 +13,7 @@ class CustomSqliteDialect extends SqliteDialect {
 
 export class SqliteLocalConnection extends SqliteConnection {
    constructor(private database: SqliteDatabase) {
-      const plugins = [new DeserializeJsonValuesPlugin()];
+      const plugins = [new ParseJSONResultsPlugin()];
       const kysely = new Kysely({
          dialect: new CustomSqliteDialect({ database }),
          plugins

app/src/data/entities/Entity.ts

@@ -98,8 +98,8 @@ export class Entity<
 
    getDefaultSort() {
       return {
-         by: this.config.sort_field,
-         dir: this.config.sort_dir
+         by: this.config.sort_field ?? "id",
+         dir: this.config.sort_dir ?? "asc"
       };
    }
 
@@ -140,7 +140,7 @@ export class Entity<
       return this.fields.find((field) => field.name === name);
    }
 
-   __experimental_replaceField(name: string, field: Field) {
+   __replaceField(name: string, field: Field) {
       const index = this.fields.findIndex((f) => f.name === name);
       if (index === -1) {
          throw new Error(`Field "${name}" not found on entity "${this.name}"`);
@@ -192,14 +192,41 @@ export class Entity<
       this.data = data;
    }
 
-   isValidData(data: EntityData, context: TActionContext, explain?: boolean): boolean {
+   // @todo: add tests
+   isValidData(
+      data: EntityData,
+      context: TActionContext,
+      options?: {
+         explain?: boolean;
+         ignoreUnknown?: boolean;
+      }
+   ): boolean {
+      if (typeof data !== "object") {
+         if (options?.explain) {
+            throw new Error(`Entity "${this.name}" data must be an object`);
+         }
+      }
+
       const fields = this.getFillableFields(context, false);
-      //const fields = this.fields;
-      //console.log("data", data);
+
+      if (options?.ignoreUnknown !== true) {
+         const field_names = fields.map((f) => f.name);
+         const given_keys = Object.keys(data);
+         const unknown_keys = given_keys.filter((key) => !field_names.includes(key));
+
+         if (unknown_keys.length > 0) {
+            if (options?.explain) {
+               throw new Error(
+                  `Entity "${this.name}" data must only contain known keys, unknown: "${unknown_keys}"`
+               );
+            }
+         }
+      }
+
       for (const field of fields) {
          if (!field.isValid(data[field.name], context)) {
             console.log("Entity.isValidData:invalid", context, field.name, data[field.name]);
-            if (explain) {
+            if (options?.explain) {
                throw new Error(`Field "${field.name}" has invalid data: "${data[field.name]}"`);
             }
 

app/src/data/entities/EntityManager.ts

@@ -99,17 +99,30 @@ export class EntityManager<TBD extends object = DefaultDB> {
       this.entities.push(entity);
    }
 
-   entity(e: Entity | keyof TBD | string): Entity {
-      let entity: Entity | undefined;
-      if (typeof e === "string") {
-         entity = this.entities.find((entity) => entity.name === e);
-      } else if (e instanceof Entity) {
-         entity = e;
+   __replaceEntity(entity: Entity, name: string | undefined = entity.name) {
+      const entityIndex = this._entities.findIndex((e) => e.name === name);
+
+      if (entityIndex === -1) {
+         throw new Error(`Entity "${name}" not found and cannot be replaced`);
       }
 
+      this._entities[entityIndex] = entity;
+
+      // caused issues because this.entity() was using a reference (for when initial config was given)
+   }
+
+   entity<Silent extends true | false = false>(
+      e: Entity | keyof TBD | string,
+      silent?: Silent
+   ): Silent extends true ? Entity | undefined : Entity {
+      // make sure to always retrieve by name
+      const entity = this.entities.find((entity) =>
+         e instanceof Entity ? entity.name === e.name : entity.name === e
+      );
+
       if (!entity) {
-         // @ts-ignore
-         throw new EntityNotDefinedException(e instanceof Entity ? e.name : e);
+         if (silent === true) return undefined as any;
+         throw new EntityNotDefinedException(e instanceof Entity ? e.name : (e as string));
       }
 
       return entity;

app/src/data/entities/Mutator.ts

@@ -132,14 +132,17 @@ export class Mutator<
          throw new Error(`Creation of system entity "${entity.name}" is disabled`);
       }
 
-      // @todo: establish the original order from "data"
+      const result = await this.emgr.emit(
+         new Mutator.Events.MutatorInsertBefore({ entity, data: data as any })
+      );
+
+      // if listener returned, take what's returned
+      const _data = result.returned ? result.params.data : data;
       const validatedData = {
          ...entity.getDefaultObject(),
-         ...(await this.getValidatedData(data, "create"))
+         ...(await this.getValidatedData(_data, "create"))
       };
 
-      await this.emgr.emit(new Mutator.Events.MutatorInsertBefore({ entity, data: validatedData }));
-
       // check if required fields are present
       const required = entity.getRequiredFields();
       for (const field of required) {
@@ -169,16 +172,17 @@ export class Mutator<
          throw new Error("ID must be provided for update");
       }
 
-      const validatedData = await this.getValidatedData(data, "update");
-
-      await this.emgr.emit(
+      const result = await this.emgr.emit(
          new Mutator.Events.MutatorUpdateBefore({
             entity,
             entityId: id,
-            data: validatedData as any
+            data
          })
       );
 
+      const _data = result.returned ? result.params.data : data;
+      const validatedData = await this.getValidatedData(_data, "update");
+
       const query = this.conn
          .updateTable(entity.name)
          .set(validatedData as any)

app/src/data/entities/query/Repository.ts

@@ -58,14 +58,14 @@ export class Repository<TBD extends object = DefaultDB, TB extends keyof TBD = a
    }
 
    private cloneFor(entity: Entity) {
-      return new Repository(this.em, entity, this.emgr);
+      return new Repository(this.em, this.em.entity(entity), this.emgr);
    }
 
    private get conn() {
       return this.em.connection.kysely;
    }
 
-   private getValidOptions(options?: Partial<RepoQuery>): RepoQuery {
+   getValidOptions(options?: Partial<RepoQuery>): RepoQuery {
       const entity = this.entity;
       // @todo: if not cloned deep, it will keep references and error if multiple requests come in
       const validated = {
@@ -94,23 +94,19 @@ export class Repository<TBD extends object = DefaultDB, TB extends keyof TBD = a
          if (invalid.length > 0) {
             throw new InvalidSearchParamsException(
                `Invalid select field(s): ${invalid.join(", ")}`
-            );
+            ).context({
+               entity: entity.name,
+               valid: validated.select
+            });
          }
 
          validated.select = options.select;
       }
 
-      if (options.with && options.with.length > 0) {
-         for (const entry of options.with) {
-            const related = this.em.relationOf(entity.name, entry);
-            if (!related) {
-               throw new InvalidSearchParamsException(
-                  `WITH: "${entry}" is not a relation of "${entity.name}"`
-               );
-            }
-
-            validated.with.push(entry);
-         }
+      if (options.with) {
+         const depth = WithBuilder.validateWiths(this.em, entity.name, options.with);
+         // @todo: determine allowed depth
+         validated.with = options.with;
       }
 
       if (options.join && options.join.length > 0) {
@@ -232,43 +228,79 @@ export class Repository<TBD extends object = DefaultDB, TB extends keyof TBD = a
       return { ...response, data: data[0]! };
    }
 
-   private buildQuery(
+   addOptionsToQueryBuilder(
+      _qb?: RepositoryQB,
       _options?: Partial<RepoQuery>,
-      exclude_options: (keyof RepoQuery)[] = []
-   ): { qb: RepositoryQB; options: RepoQuery } {
+      config?: {
+         validate?: boolean;
+         ignore?: (keyof RepoQuery)[];
+         alias?: string;
+         defaults?: Pick<RepoQuery, "limit" | "offset">;
+      }
+   ) {
       const entity = this.entity;
-      const options = this.getValidOptions(_options);
+      let qb = _qb ?? (this.conn.selectFrom(entity.name) as RepositoryQB);
 
-      const alias = entity.name;
+      const options = config?.validate !== false ? this.getValidOptions(_options) : _options;
+      if (!options) return qb;
+
+      const alias = config?.alias ?? entity.name;
       const aliased = (field: string) => `${alias}.${field}`;
-      let qb = this.conn
-         .selectFrom(entity.name)
-         .select(entity.getAliasedSelectFrom(options.select, alias));
+      const ignore = config?.ignore ?? [];
+      const defaults = {
+         limit: 10,
+         offset: 0,
+         ...config?.defaults
+      };
 
-      //console.log("build query options", options);
-      if (!exclude_options.includes("with") && options.with) {
+      /*console.log("build query options", {
+         entity: entity.name,
+         options,
+         config
+      });*/
+
+      if (!ignore.includes("select") && options.select) {
+         qb = qb.select(entity.getAliasedSelectFrom(options.select, alias));
+      }
+
+      if (!ignore.includes("with") && options.with) {
          qb = WithBuilder.addClause(this.em, qb, entity, options.with);
       }
 
-      if (!exclude_options.includes("join") && options.join) {
+      if (!ignore.includes("join") && options.join) {
          qb = JoinBuilder.addClause(this.em, qb, entity, options.join);
       }
 
       // add where if present
-      if (!exclude_options.includes("where") && options.where) {
+      if (!ignore.includes("where") && options.where) {
          qb = WhereBuilder.addClause(qb, options.where);
       }
 
-      if (!exclude_options.includes("limit")) qb = qb.limit(options.limit);
-      if (!exclude_options.includes("offset")) qb = qb.offset(options.offset);
+      if (!ignore.includes("limit")) qb = qb.limit(options.limit ?? defaults.limit);
+      if (!ignore.includes("offset")) qb = qb.offset(options.offset ?? defaults.offset);
 
       // sorting
-      if (!exclude_options.includes("sort")) {
-         qb = qb.orderBy(aliased(options.sort.by), options.sort.dir);
+      if (!ignore.includes("sort")) {
+         qb = qb.orderBy(aliased(options.sort?.by ?? "id"), options.sort?.dir ?? "asc");
       }
 
-      //console.log("options", { _options, options, exclude_options });
-      return { qb, options };
+      return qb as RepositoryQB;
+   }
+
+   private buildQuery(
+      _options?: Partial<RepoQuery>,
+      ignore: (keyof RepoQuery)[] = []
+   ): { qb: RepositoryQB; options: RepoQuery } {
+      const entity = this.entity;
+      const options = this.getValidOptions(_options);
+
+      return {
+         qb: this.addOptionsToQueryBuilder(undefined, options, {
+            ignore,
+            alias: entity.name
+         }),
+         options
+      };
    }
 
    async findId(

app/src/data/entities/query/WhereBuilder.ts

@@ -30,7 +30,7 @@ function key(e: unknown): string {
    return e as string;
 }
 
-const expressions: TExpression<any, any, any>[] = [
+const expressions = [
    exp(
       "$eq",
       (v: Primitive) => isPrimitive(v),

app/src/data/entities/query/WithBuilder.ts

@@ -1,42 +1,82 @@
+import { isObject } from "core/utils";
+import type { KyselyJsonFrom, RepoQuery } from "data";
+import { InvalidSearchParamsException } from "data/errors";
 import type { Entity, EntityManager, RepositoryQB } from "../../entities";
 
 export class WithBuilder {
-   private static buildClause(
+   static addClause(
       em: EntityManager<any>,
       qb: RepositoryQB,
       entity: Entity,
-      withString: string
+      withs: RepoQuery["with"]
    ) {
-      const relation = em.relationOf(entity.name, withString);
-      if (!relation) {
-         throw new Error(`Relation "${withString}" not found`);
+      if (!withs || !isObject(withs)) {
+         console.warn(`'withs' undefined or invalid, given: ${JSON.stringify(withs)}`);
+         return qb;
       }
 
-      const cardinality = relation.ref(withString).cardinality;
-      //console.log("with--builder", { entity: entity.name, withString, cardinality });
-
       const fns = em.connection.fn;
-      const jsonFrom = cardinality === 1 ? fns.jsonObjectFrom : fns.jsonArrayFrom;
-
-      if (!jsonFrom) {
-         throw new Error("Connection does not support jsonObjectFrom/jsonArrayFrom");
-      }
-
-      try {
-         return relation.buildWith(entity, qb, jsonFrom, withString);
-      } catch (e) {
-         throw new Error(`Could not build "with" relation "${withString}": ${(e as any).message}`);
-      }
-   }
-
-   static addClause(em: EntityManager<any>, qb: RepositoryQB, entity: Entity, withs: string[]) {
-      if (withs.length === 0) return qb;
-
       let newQb = qb;
-      for (const entry of withs) {
-         newQb = WithBuilder.buildClause(em, newQb, entity, entry);
+
+      for (const [ref, query] of Object.entries(withs)) {
+         const relation = em.relationOf(entity.name, ref);
+         if (!relation) {
+            throw new Error(`Relation "${entity.name}<>${ref}" not found`);
+         }
+         const cardinality = relation.ref(ref).cardinality;
+         const jsonFrom: KyselyJsonFrom =
+            cardinality === 1 ? fns.jsonObjectFrom : fns.jsonArrayFrom;
+         if (!jsonFrom) {
+            throw new Error("Connection does not support jsonObjectFrom/jsonArrayFrom");
+         }
+
+         const other = relation.other(entity);
+         newQb = newQb.select((eb) => {
+            let subQuery = relation.buildWith(entity, ref)(eb);
+            if (query) {
+               subQuery = em.repo(other.entity).addOptionsToQueryBuilder(subQuery, query as any, {
+                  ignore: ["with", "join", cardinality === 1 ? "limit" : undefined].filter(
+                     Boolean
+                  ) as any
+               });
+            }
+
+            if (query.with) {
+               subQuery = WithBuilder.addClause(em, subQuery, other.entity, query.with as any);
+            }
+
+            return jsonFrom(subQuery).as(other.reference);
+         });
       }
 
       return newQb;
    }
+
+   static validateWiths(em: EntityManager<any>, entity: string, withs: RepoQuery["with"]) {
+      let depth = 0;
+      if (!withs || !isObject(withs)) {
+         withs && console.warn(`'withs' invalid, given: ${JSON.stringify(withs)}`);
+         return depth;
+      }
+
+      const child_depths: number[] = [];
+      for (const [ref, query] of Object.entries(withs)) {
+         const related = em.relationOf(entity, ref);
+         if (!related) {
+            throw new InvalidSearchParamsException(
+               `WITH: "${ref}" is not a relation of "${entity}"`
+            );
+         }
+         depth++;
+
+         if ("with" in query) {
+            child_depths.push(WithBuilder.validateWiths(em, ref, query.with as any));
+         }
+      }
+      if (child_depths.length > 0) {
+         depth += Math.max(...child_depths);
+      }
+
+      return depth;
+   }
 }

app/src/data/events/index.ts

@@ -1,20 +1,48 @@
 import type { PrimaryFieldType } from "core";
-import { Event } from "core/events";
+import { Event, InvalidEventReturn } from "core/events";
 import type { Entity, EntityData } from "../entities";
 import type { RepoQuery } from "../server/data-query-impl";
 
-export class MutatorInsertBefore extends Event<{ entity: Entity; data: EntityData }> {
+export class MutatorInsertBefore extends Event<{ entity: Entity; data: EntityData }, EntityData> {
    static override slug = "mutator-insert-before";
+
+   override validate(data: EntityData) {
+      const { entity } = this.params;
+      if (!entity.isValidData(data, "create")) {
+         throw new InvalidEventReturn("EntityData", "invalid");
+      }
+
+      return this.clone({
+         entity,
+         data
+      });
+   }
 }
 export class MutatorInsertAfter extends Event<{ entity: Entity; data: EntityData }> {
    static override slug = "mutator-insert-after";
 }
-export class MutatorUpdateBefore extends Event<{
-   entity: Entity;
-   entityId: PrimaryFieldType;
-   data: EntityData;
-}> {
+export class MutatorUpdateBefore extends Event<
+   {
+      entity: Entity;
+      entityId: PrimaryFieldType;
+      data: EntityData;
+   },
+   EntityData
+> {
    static override slug = "mutator-update-before";
+
+   override validate(data: EntityData) {
+      const { entity, ...rest } = this.params;
+      if (!entity.isValidData(data, "update")) {
+         throw new InvalidEventReturn("EntityData", "invalid");
+      }
+
+      return this.clone({
+         ...rest,
+         entity,
+         data
+      });
+   }
 }
 export class MutatorUpdateAfter extends Event<{
    entity: Entity;

app/src/data/fields/Field.ts

@@ -12,6 +12,9 @@ import type { HTMLInputTypeAttribute, InputHTMLAttributes } from "react";
 import type { EntityManager } from "../entities";
 import { InvalidFieldConfigException, TransformPersistFailedException } from "../errors";
 
+// @todo: contexts need to be reworked
+// e.g. "table" is irrelevant, because if read is not given, it fails
+
 export const ActionContext = ["create", "read", "update", "delete"] as const;
 export type TActionContext = (typeof ActionContext)[number];
 
@@ -157,8 +160,12 @@ export abstract class Field<
       return this.config.virtual ?? false;
    }
 
-   getLabel(): string {
-      return this.config.label ?? snakeToPascalWithSpaces(this.name);
+   getLabel(options?: { fallback?: boolean }): string | undefined {
+      return this.config.label
+         ? this.config.label
+         : options?.fallback !== false
+           ? snakeToPascalWithSpaces(this.name)
+           : undefined;
    }
 
    getDescription(): string | undefined {

app/src/data/helper.ts

@@ -1,4 +1,4 @@
-import type { EntityData, Field } from "data";
+import type { EntityData, EntityManager, Field } from "data";
 import { transform } from "lodash-es";
 
 export function getDefaultValues(fields: Field[], data: EntityData): EntityData {
@@ -48,3 +48,23 @@ export function getChangeSet(
       {} as typeof formData
    );
 }
+
+export function readableEmJson(_em: EntityManager) {
+   return {
+      entities: _em.entities.map((e) => ({
+         name: e.name,
+         fields: e.fields.map((f) => f.name),
+         type: e.type
+      })),
+      indices: _em.indices.map((i) => ({
+         name: i.name,
+         entity: i.entity.name,
+         fields: i.fields.map((f) => f.name),
+         unique: i.unique
+      })),
+      relations: _em.relations.all.map((r) => ({
+         name: r.getName(),
+         ...r.toJSON()
+      }))
+   };
+}

app/src/data/index.ts

@@ -8,6 +8,7 @@ export * from "./prototype";
 
 export {
    type RepoQuery,
+   type RepoQueryIn,
    defaultQuerySchema,
    querySchema,
    whereSchema

app/src/data/prototype/index.ts

@@ -272,18 +272,22 @@ class EntityManagerPrototype<Entities extends Record<string, Entity>> extends En
    }
 }
 
-type Chained<Fn extends (...args: any[]) => any, Rt = ReturnType<Fn>> = <E extends Entity>(
-   e: E
-) => {
-   [K in keyof Rt]: Rt[K] extends (...args: any[]) => any
-      ? (...args: Parameters<Rt[K]>) => Rt
+type Chained<R extends Record<string, (...args: any[]) => any>> = {
+   [K in keyof R]: R[K] extends (...args: any[]) => any
+      ? (...args: Parameters<R[K]>) => Chained<R>
       : never;
 };
+type ChainedFn<
+   Fn extends (...args: any[]) => Record<string, (...args: any[]) => any>,
+   Return extends ReturnType<Fn> = ReturnType<Fn>
+> = (e: Entity) => {
+   [K in keyof Return]: (...args: Parameters<Return[K]>) => Chained<Return>;
+};
 
 export function em<Entities extends Record<string, Entity>>(
    entities: Entities,
    schema?: (
-      fns: { relation: Chained<typeof relation>; index: Chained<typeof index> },
+      fns: { relation: ChainedFn<typeof relation>; index: ChainedFn<typeof index> },
       entities: Entities
    ) => void
 ) {

app/src/data/relations/EntityRelation.ts

@@ -1,5 +1,5 @@
 import { type Static, Type, parse } from "core/utils";
-import type { SelectQueryBuilder } from "kysely";
+import type { ExpressionBuilder, SelectQueryBuilder } from "kysely";
 import type { Entity, EntityData, EntityManager } from "../entities";
 import {
    type EntityRelationAnchor,
@@ -67,10 +67,8 @@ export abstract class EntityRelation<
     */
    abstract buildWith(
       entity: Entity,
-      qb: KyselyQueryBuilder,
-      jsonFrom: KyselyJsonFrom,
       reference: string
-   ): KyselyQueryBuilder;
+   ): (eb: ExpressionBuilder<any, any>) => KyselyQueryBuilder;
 
    abstract buildJoin(
       entity: Entity,

app/src/data/relations/ManyToManyRelation.ts

@@ -1,4 +1,5 @@
 import { type Static, Type } from "core/utils";
+import type { ExpressionBuilder } from "kysely";
 import { Entity, type EntityManager } from "../entities";
 import { type Field, PrimaryField, VirtualField } from "../fields";
 import type { RepoQuery } from "../server/data-query-impl";
@@ -123,7 +124,7 @@ export class ManyToManyRelation extends EntityRelation<typeof ManyToManyRelation
          .groupBy(groupBy);
    }
 
-   buildWith(entity: Entity, qb: KyselyQueryBuilder, jsonFrom: KyselyJsonFrom) {
+   buildWith(entity: Entity) {
       if (!this.em) {
          throw new Error("EntityManager not set, can't build");
       }
@@ -138,7 +139,29 @@ export class ManyToManyRelation extends EntityRelation<typeof ManyToManyRelation
          (f) => !(f instanceof RelationField || f instanceof PrimaryField)
       );
 
-      return qb.select((eb) => {
+      return (eb: ExpressionBuilder<any, any>) =>
+         eb
+            .selectFrom(other.entity.name)
+            .select((eb2) => {
+               const select: any[] = other.entity.getSelect(other.entity.name);
+               if (additionalFields.length > 0) {
+                  const conn = this.connectionEntity.name;
+                  select.push(
+                     jsonBuildObject(
+                        Object.fromEntries(
+                           additionalFields.map((f) => [f.name, eb2.ref(`${conn}.${f.name}`)])
+                        )
+                     ).as(this.connectionTableMappedName)
+                  );
+               }
+
+               return select;
+            })
+            .whereRef(entityRef, "=", otherRef)
+            .innerJoin(...join)
+            .limit(limit);
+
+      /*return qb.select((eb) => {
          const select: any[] = other.entity.getSelect(other.entity.name);
          // @todo: also add to find by references
          if (additionalFields.length > 0) {
@@ -160,7 +183,7 @@ export class ManyToManyRelation extends EntityRelation<typeof ManyToManyRelation
                .innerJoin(...join)
                .limit(limit)
          ).as(other.reference);
-      });
+      });*/
    }
 
    initialize(em: EntityManager<any>) {

app/src/data/relations/ManyToOneRelation.ts

@@ -1,6 +1,7 @@
 import type { PrimaryFieldType } from "core";
 import { snakeToPascalWithSpaces } from "core/utils";
 import { type Static, Type } from "core/utils";
+import type { ExpressionBuilder } from "kysely";
 import type { Entity, EntityManager } from "../entities";
 import type { RepoQuery } from "../server/data-query-impl";
 import { EntityRelation, type KyselyJsonFrom, type KyselyQueryBuilder } from "./EntityRelation";
@@ -155,23 +156,14 @@ export class ManyToOneRelation extends EntityRelation<typeof ManyToOneRelation.s
       return qb.innerJoin(self.entity.name, entityRef, otherRef).groupBy(groupBy);
    }
 
-   buildWith(entity: Entity, qb: KyselyQueryBuilder, jsonFrom: KyselyJsonFrom, reference: string) {
+   buildWith(entity: Entity, reference: string) {
       const { self, entityRef, otherRef, relationRef } = this.queryInfo(entity, reference);
-      const limit =
-         self.cardinality === 1
-            ? 1
-            : this.config.with_limit ?? ManyToOneRelation.DEFAULTS.with_limit;
-      //console.log("buildWith", entity.name, reference, { limit });
 
-      return qb.select((eb) =>
-         jsonFrom(
-            eb
-               .selectFrom(`${self.entity.name} as ${relationRef}`)
-               .select(self.entity.getSelect(relationRef))
-               .whereRef(entityRef, "=", otherRef)
-               .limit(limit)
-         ).as(relationRef)
-      );
+      return (eb: ExpressionBuilder<any, any>) =>
+         eb
+            .selectFrom(`${self.entity.name} as ${relationRef}`)
+            .whereRef(entityRef, "=", otherRef)
+            .$if(self.cardinality === 1, (qb) => qb.limit(1));
    }
 
    /**

app/src/data/relations/PolymorphicRelation.ts

@@ -1,4 +1,5 @@
 import { type Static, Type } from "core/utils";
+import type { ExpressionBuilder } from "kysely";
 import type { Entity, EntityManager } from "../entities";
 import { NumberField, TextField } from "../fields";
 import type { RepoQuery } from "../server/data-query-impl";
@@ -87,20 +88,15 @@ export class PolymorphicRelation extends EntityRelation<typeof PolymorphicRelati
       };
    }
 
-   buildWith(entity: Entity, qb: KyselyQueryBuilder, jsonFrom: KyselyJsonFrom) {
+   buildWith(entity: Entity) {
       const { other, whereLhs, reference, entityRef, otherRef } = this.queryInfo(entity);
-      const limit = other.cardinality === 1 ? 1 : 5;
 
-      return qb.select((eb) =>
-         jsonFrom(
-            eb
-               .selectFrom(other.entity.name)
-               .select(other.entity.getSelect(other.entity.name))
-               .where(whereLhs, "=", reference)
-               .whereRef(entityRef, "=", otherRef)
-               .limit(limit)
-         ).as(other.reference)
-      );
+      return (eb: ExpressionBuilder<any, any>) =>
+         eb
+            .selectFrom(other.entity.name)
+            .where(whereLhs, "=", reference)
+            .whereRef(entityRef, "=", otherRef)
+            .$if(other.cardinality === 1, (qb) => qb.limit(1));
    }
 
    override isListableFor(entity: Entity): boolean {

app/src/data/server/data-query-impl.ts

@@ -1,3 +1,4 @@
+import type { TThis } from "@sinclair/typebox";
 import {
    type SchemaOptions,
    type Static,
@@ -6,8 +7,7 @@ import {
    Type,
    Value
 } from "core/utils";
-import type { Simplify } from "type-fest";
-import { WhereBuilder } from "../entities";
+import { WhereBuilder, type WhereQuery } from "../entities";
 
 const NumberOrString = (options: SchemaOptions = {}) =>
    Type.Transform(Type.Union([Type.Number(), Type.String()], options))
@@ -15,25 +15,31 @@ const NumberOrString = (options: SchemaOptions = {}) =>
       .Encode(String);
 
 const limit = NumberOrString({ default: 10 });
-
 const offset = NumberOrString({ default: 0 });
 
-// @todo: allow "id" and "-id"
+const sort_default = { by: "id", dir: "asc" };
 const sort = Type.Transform(
    Type.Union(
       [Type.String(), Type.Object({ by: Type.String(), dir: StringEnum(["asc", "desc"]) })],
       {
-         default: { by: "id", dir: "asc" }
+         default: sort_default
       }
    )
 )
-   .Decode((value) => {
+   .Decode((value): { by: string; dir: "asc" | "desc" } => {
       if (typeof value === "string") {
-         return JSON.parse(value);
+         if (/^-?[a-zA-Z_][a-zA-Z0-9_.]*$/.test(value)) {
+            const dir = value[0] === "-" ? "desc" : "asc";
+            return { by: dir === "desc" ? value.slice(1) : value, dir } as any;
+         } else if (/^{.*}$/.test(value)) {
+            return JSON.parse(value) as any;
+         }
+
+         return sort_default as any;
       }
-      return value;
+      return value as any;
    })
-   .Encode(JSON.stringify);
+   .Encode((value) => value);
 
 const stringArray = Type.Transform(
    Type.Union([Type.String(), Type.Array(Type.String())], { default: [] })
@@ -57,21 +63,63 @@ export const whereSchema = Type.Transform(
    })
    .Encode(JSON.stringify);
 
-export const querySchema = Type.Object(
-   {
-      limit: Type.Optional(limit),
-      offset: Type.Optional(offset),
-      sort: Type.Optional(sort),
-      select: Type.Optional(stringArray),
-      with: Type.Optional(stringArray),
-      join: Type.Optional(stringArray),
-      where: Type.Optional(whereSchema)
-   },
-   {
-      additionalProperties: false
+export type RepoWithSchema = Record<
+   string,
+   Omit<RepoQueryIn, "with"> & {
+      with?: unknown;
    }
+>;
+
+export const withSchema = <TSelf extends TThis>(Self: TSelf) =>
+   Type.Transform(Type.Union([stringArray, Type.Record(Type.String(), Self)]))
+      .Decode((value) => {
+         let _value = typeof value === "string" ? [value] : value;
+
+         if (Array.isArray(value)) {
+            if (!value.every((v) => typeof v === "string")) {
+               throw new Error("Invalid 'with' schema");
+            }
+
+            _value = value.reduce((acc, v) => {
+               acc[v] = {};
+               return acc;
+            }, {} as RepoWithSchema);
+         }
+
+         return _value as RepoWithSchema;
+      })
+      .Encode((value) => value);
+
+export const querySchema = Type.Recursive(
+   (Self) =>
+      Type.Partial(
+         Type.Object(
+            {
+               limit: limit,
+               offset: offset,
+               sort: sort,
+               select: stringArray,
+               with: withSchema(Self),
+               join: stringArray,
+               where: whereSchema
+            },
+            {
+               // @todo: determine if unknown is allowed, it's ignore anyway
+               additionalProperties: false
+            }
+         )
+      ),
+   { $id: "query-schema" }
 );
 
-export type RepoQueryIn = Static<typeof querySchema>;
+export type RepoQueryIn = {
+   limit?: number;
+   offset?: number;
+   sort?: string | { by: string; dir: "asc" | "desc" };
+   select?: string[];
+   with?: string[] | Record<string, RepoQueryIn>;
+   join?: string[];
+   where?: WhereQuery;
+};
 export type RepoQuery = Required<StaticDecode<typeof querySchema>>;
 export const defaultQuerySchema = Value.Default(querySchema, {}) as RepoQuery;

app/src/flows/AppFlows.ts

@@ -12,6 +12,18 @@ export type { TAppFlowTaskSchema } from "./flows-schema";
 export class AppFlows extends Module<typeof flowsConfigSchema> {
    private flows: Record<string, Flow> = {};
 
+   getSchema() {
+      return flowsConfigSchema;
+   }
+
+   private getFlowInfo(flow: Flow) {
+      return {
+         ...flow.toJSON(),
+         tasks: flow.tasks.length,
+         connections: flow.connections
+      };
+   }
+
    override async build() {
       //console.log("building flows", this.config);
       const flows = transformObject(this.config.flows, (flowConfig, name) => {
@@ -67,15 +79,10 @@ export class AppFlows extends Module<typeof flowsConfigSchema> {
       this.setBuilt();
    }
 
-   getSchema() {
-      return flowsConfigSchema;
-   }
-
-   private getFlowInfo(flow: Flow) {
+   override toJSON() {
       return {
-         ...flow.toJSON(),
-         tasks: flow.tasks.length,
-         connections: flow.connections
+         ...this.config,
+         flows: transformObject(this.flows, (flow) => flow.toJSON())
       };
    }
 }

app/src/flows/flows-schema.ts

@@ -62,7 +62,7 @@ export const flowSchema = Type.Object(
    {
       trigger: Type.Union(Object.values(triggerSchemaObject)),
       tasks: Type.Optional(StringRecord(Type.Union(Object.values(taskSchemaObject)))),
-      connections: Type.Optional(StringRecord(connectionSchema, { default: {} })),
+      connections: Type.Optional(StringRecord(connectionSchema)),
       start_task: Type.Optional(Type.String()),
       responding_task: Type.Optional(Type.String())
    },

app/src/flows/flows/Flow.ts

@@ -162,8 +162,8 @@ export class Flow {
          trigger: this.trigger.toJSON(),
          tasks: Object.fromEntries(this.tasks.map((t) => [t.name, t.toJSON()])),
          connections: Object.fromEntries(this.connections.map((c) => [c.id, c.toJSON()])),
-         start_task: this.startTask.name,
-         responding_task: this.respondingTask ? this.respondingTask.name : null
+         start_task: this.startTask?.name,
+         responding_task: this.respondingTask?.name
       };
    }
 

app/src/flows/tasks/TaskConnection.ts

@@ -1,4 +1,4 @@
-import { uuid } from "core/utils";
+import { objectCleanEmpty, uuid } from "core/utils";
 import { get } from "lodash-es";
 import type { Task, TaskResult } from "./Task";
 
@@ -34,14 +34,14 @@ export class TaskConnection {
    }
 
    toJSON() {
-      return {
+      return objectCleanEmpty({
          source: this.source.name,
          target: this.target.name,
          config: {
             ...this.config,
             condition: this.config.condition?.toJSON()
          }
-      };
+      });
    }
 }
 

app/src/index.ts

@@ -9,7 +9,11 @@ export {
    type ModuleBuildContext
 } from "./modules/ModuleManager";
 
+export * as middlewares from "modules/middlewares";
 export { registries } from "modules/registries";
 
 export type * from "./adapter";
 export { Api, type ApiOptions } from "./Api";
+
+export type { MediaFieldSchema } from "media/AppMedia";
+export type { UserFieldSchema } from "auth/AppAuth";

app/src/media/AppMedia.ts

@@ -1,8 +1,17 @@
 import type { PrimaryFieldType } from "core";
-import { EntityIndex, type EntityManager } from "data";
+import { type Entity, EntityIndex, type EntityManager } from "data";
 import { type FileUploadedEventData, Storage, type StorageAdapter } from "media";
 import { Module } from "modules/Module";
-import { type FieldSchema, boolean, datetime, entity, json, number, text } from "../data/prototype";
+import {
+   type FieldSchema,
+   boolean,
+   datetime,
+   em,
+   entity,
+   json,
+   number,
+   text
+} from "../data/prototype";
 import { MediaController } from "./api/MediaController";
 import { ADAPTERS, buildMediaSchema, type mediaConfigSchema, registry } from "./media-schema";
 
@@ -38,18 +47,14 @@ export class AppMedia extends Module<typeof mediaConfigSchema> {
          this.setupListeners();
          this.ctx.server.route(this.basepath, new MediaController(this).getController());
 
-         // @todo: add check for media entity
-         const mediaEntity = this.getMediaEntity();
-         if (!this.ctx.em.hasEntity(mediaEntity)) {
-            this.ctx.em.addEntity(mediaEntity);
-         }
+         const media = this.getMediaEntity(true);
+         this.ensureSchema(
+            em({ [media.name as "media"]: media }, ({ index }, { media }) => {
+               index(media).on(["path"], true).on(["reference"]);
+            })
+         );
 
-         const pathIndex = new EntityIndex(mediaEntity, [mediaEntity.field("path")!], true);
-         if (!this.ctx.em.hasIndex(pathIndex)) {
-            this.ctx.em.addIndex(pathIndex);
-         }
-
-         // @todo: check indices
+         this.setBuilt();
       } catch (e) {
          console.error(e);
          throw new Error(
@@ -94,13 +99,13 @@ export class AppMedia extends Module<typeof mediaConfigSchema> {
       metadata: json()
    };
 
-   getMediaEntity() {
+   getMediaEntity(forceCreate?: boolean): Entity<"media", typeof AppMedia.mediaFields> {
       const entity_name = this.config.entity_name;
-      if (!this.em.hasEntity(entity_name)) {
-         return entity(entity_name, AppMedia.mediaFields, undefined, "system");
+      if (forceCreate || !this.em.hasEntity(entity_name)) {
+         return entity(entity_name as "media", AppMedia.mediaFields, undefined, "system");
       }
 
-      return this.em.entity(entity_name);
+      return this.em.entity(entity_name) as any;
    }
 
    get em(): EntityManager {

app/src/media/api/MediaApi.ts

@@ -1,5 +1,5 @@
 import { type BaseModuleApiOptions, ModuleApi, type PrimaryFieldType } from "modules/ModuleApi";
-import type { FileWithPath } from "ui/modules/media/components/dropzone/file-selector";
+import type { FileWithPath } from "ui/elements/media/file-selector";
 
 export type MediaApiOptions = BaseModuleApiOptions & {};
 

app/src/media/api/MediaController.ts

@@ -1,10 +1,9 @@
-import { type ClassController, tbValidator as tb } from "core";
+import { tbValidator as tb } from "core";
 import { Type } from "core/utils";
-import { Hono } from "hono";
 import { bodyLimit } from "hono/body-limit";
 import type { StorageAdapter } from "media";
-import { StorageEvents } from "media";
-import { getRandomizedFilename } from "media";
+import { StorageEvents, getRandomizedFilename } from "media";
+import { Controller } from "modules/Controller";
 import type { AppMedia } from "../AppMedia";
 import { MediaField } from "../MediaField";
 
@@ -12,8 +11,10 @@ const booleanLike = Type.Transform(Type.String())
    .Decode((v) => v === "1")
    .Encode((v) => (v ? "1" : "0"));
 
-export class MediaController implements ClassController {
-   constructor(private readonly media: AppMedia) {}
+export class MediaController extends Controller {
+   constructor(private readonly media: AppMedia) {
+      super();
+   }
 
    private getStorageAdapter(): StorageAdapter {
       return this.getStorage().getAdapter();
@@ -23,11 +24,11 @@ export class MediaController implements ClassController {
       return this.media.storage;
    }
 
-   getController(): Hono<any> {
+   override getController() {
       // @todo: multiple providers?
       // @todo: implement range requests
-
-      const hono = new Hono();
+      const { auth } = this.middlewares;
+      const hono = this.create().use(auth());
 
       // get files list (temporary)
       hono.get("/files", async (c) => {
@@ -107,7 +108,7 @@ export class MediaController implements ClassController {
                return c.json({ error: `Invalid field "${field_name}"` }, 400);
             }
 
-            const mediaEntity = this.media.getMediaEntity();
+            const media_entity = this.media.getMediaEntity().name as "media";
             const reference = `${entity_name}.${field_name}`;
             const mediaRef = {
                scope: field_name,
@@ -117,11 +118,10 @@ export class MediaController implements ClassController {
 
             // check max items
             const max_items = field.getMaxItems();
-            const ids_to_delete: number[] = [];
-            const id_field = mediaEntity.getPrimaryField().name;
+            const paths_to_delete: string[] = [];
             if (max_items) {
                const { overwrite } = c.req.valid("query");
-               const { count } = await this.media.em.repository(mediaEntity).count(mediaRef);
+               const { count } = await this.media.em.repository(media_entity).count(mediaRef);
 
                // if there are more than or equal to max items
                if (count >= max_items) {
@@ -140,18 +140,18 @@ export class MediaController implements ClassController {
                   }
 
                   // collect items to delete
-                  const deleteRes = await this.media.em.repo(mediaEntity).findMany({
-                     select: [id_field],
+                  const deleteRes = await this.media.em.repo(media_entity).findMany({
+                     select: ["path"],
                      where: mediaRef,
                      sort: {
-                        by: id_field,
+                        by: "id",
                         dir: "asc"
                      },
                      limit: count - max_items + 1
                   });
 
                   if (deleteRes.data && deleteRes.data.length > 0) {
-                     deleteRes.data.map((item) => ids_to_delete.push(item[id_field]));
+                     deleteRes.data.map((item) => paths_to_delete.push(item.path));
                   }
                }
             }
@@ -169,7 +169,7 @@ export class MediaController implements ClassController {
             const file_name = getRandomizedFilename(file as File);
             const info = await this.getStorage().uploadFile(file, file_name, true);
 
-            const mutator = this.media.em.mutator(mediaEntity);
+            const mutator = this.media.em.mutator(media_entity);
             mutator.__unstable_toggleSystemEntityCreation(false);
             const result = await mutator.insertOne({
                ...this.media.uploadedEventDataToMediaPayload(info),
@@ -178,10 +178,11 @@ export class MediaController implements ClassController {
             mutator.__unstable_toggleSystemEntityCreation(true);
 
             // delete items if needed
-            if (ids_to_delete.length > 0) {
-               await this.media.em
-                  .mutator(mediaEntity)
-                  .deleteWhere({ [id_field]: { $in: ids_to_delete } });
+            if (paths_to_delete.length > 0) {
+               // delete files from db & adapter
+               for (const path of paths_to_delete) {
+                  await this.getStorage().deleteFile(path);
+               }
             }
 
             return c.json({ ok: true, result: result.data, ...info });

app/src/media/index.ts

@@ -1,7 +1,8 @@
 import type { TObject, TString } from "@sinclair/typebox";
 import { type Constructor, Registry } from "core";
 
-export { MIME_TYPES } from "./storage/mime-types";
+//export { MIME_TYPES } from "./storage/mime-types";
+export { guess as guessMimeType } from "./storage/mime-types-tiny";
 export {
    Storage,
    type StorageAdapter,
@@ -19,7 +20,7 @@ import { type S3AdapterConfig, StorageS3Adapter } from "./storage/adapters/Stora
 export { StorageS3Adapter, type S3AdapterConfig, StorageCloudinaryAdapter, type CloudinaryConfig };
 
 export * as StorageEvents from "./storage/events";
-export { type FileUploadedEventData } from "./storage/events";
+export type { FileUploadedEventData } from "./storage/events";
 export * from "./utils";
 
 type ClassThatImplements<T> = Constructor<T> & { prototype: T };

app/src/media/media-schema.ts

@@ -1,4 +1,4 @@
-import { Const, Type, objectTransform } from "core/utils";
+import { Const, type Static, Type, objectTransform } from "core/utils";
 import { Adapters } from "media";
 import { registries } from "modules/registries";
 
@@ -47,3 +47,4 @@ export function buildMediaSchema() {
 }
 
 export const mediaConfigSchema = buildMediaSchema();
+export type TAppMediaConfig = Static<typeof mediaConfigSchema>;