Merge pull request #72 from bknd-io/fix/0.7.1

fix: popover regression & dropzone rerender lost state

.gitignore

@@ -18,6 +18,7 @@ packages/media/.env
 **/*/vite.config.ts.timestamp*
 .history
 **/*/.db/*
+**/*/.configs/*
 **/*/*.db
 **/*/*.db-shm
 **/*/*.db-wal

LICENSE.md

@@ -6,7 +6,7 @@ FSL-1.1-MIT
 
 ## Notice
 
-Copyright 2024 Webintex GmbH
+Copyright 2025 Dennis Senn
 
 ## Terms and Conditions
 

README.md

@@ -1,9 +1,15 @@
-![bknd](https://raw.githubusercontent.com/bknd-io/bknd/refs/heads/main/app/src/ui/assets/poster.png)
+[![npm version](https://img.shields.io/npm/v/bknd.svg)](https://npmjs.org/package/bknd)
+[![npm downloads](https://img.shields.io/npm/dm/bknd)](https://www.npmjs.com/package/bknd)
 
-bknd simplifies app development by providing fully functional backend for data management, 
-authentication, workflows and media. Since it's lightweight and built on Web Standards, it can 
-be deployed nearly anywhere, including running inside your framework of choice. No more 
-deploying multiple separate services!
+![bknd](https://raw.githubusercontent.com/bknd-io/bknd/refs/heads/main/docs/_assets/poster.png)
+
+<p align="center" width="100%">
+<a href="https://stackblitz.com/github/bknd-io/bknd-examples?hideExplorer=1&embed=1&view=preview&startScript=example-admin-rich&initialPath=%2Fdata%2Fschema">
+<strong>⭐ Live Demo</strong>
+</a>
+</p>
+
+bknd simplifies app development by providing a fully functional backend for database management, authentication, media and workflows. Being lightweight and built on Web Standards, it can be deployed nearly anywhere, including running inside your framework of choice. No more deploying multiple separate services!
 
 **For documentation and examples, please visit https://docs.bknd.io.**
 
@@ -11,33 +17,116 @@ deploying multiple separate services!
 > Please keep in mind that **bknd** is still under active development
 > and therefore full backward compatibility is not guaranteed before reaching v1.0.0.
 
-## ✨ Features
-**📊 Data**: Define, query, and control your data with ease. 
-  - Define entities with fields and relationships, synced directly to your database.  
-  - Supported field types: `primary`, `text`, `number`, `date`, `boolean`, `enum`, `json`, `jsonschema`.  
-  - Relationship types: `one-to-one`, `many-to-one`, `many-to-many`, and `polymorphic`.  
-  - Advanced querying with the **Repository**: filtering, sorting, pagination, and relational data handling.
-  - Seamlessly manage data with mutators and a robust event system.  
-  - Extend database functionality with batching, introspection, and support for multiple SQL dialects.
+## Size
+![gzipped size of bknd](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/index.js?compression=gzip&label=bknd)
+![gzipped size of bknd/client](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/ui/client/index.js?compression=gzip&label=bknd/client)
+![gzipped size of bknd/elements](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/ui/elements/index.js?compression=gzip&label=bknd/elements)
+![gzipped size of bknd/ui](https://img.badgesize.io/https://unpkg.com/bknd@0.6.1/dist/ui/index.js?compression=gzip&label=bknd/ui)
 
-**🔐 Auth**: Easily implement reliable authentication strategies.
-  - Built-in `user` entity with customizable fields.  
-  - Supports multiple authentication strategies:  
-    - Email/password (with hashed storage).  
-    - OAuth/OIDC (Google, GitHub, and more).  
-  - Secure JWT generation and session management.
+The size on npm is misleading, as the `bknd` package includes the backend, the ui components as well as the whole backend bundled into the cli including static assets. 
 
-**🖼️ Media**: Effortlessly manage and serve all your media files.
-  - Upload files with ease.  
-  - Adapter-based support for S3, S3-compatible storage (e.g., R2, Tigris), and Cloudinary.
+## Motivation
+Creating digital products always requires developing both the backend (the logic) and the frontend (the appearance). Building a backend from scratch demands deep knowledge in areas such as authentication and database management. Using a backend framework can speed up initial development, but it still requires ongoing effort to work within its constraints (e.g., *"how to do X with Y?"*), which can quickly slow you down. Choosing a backend system is a tough decision, as you might not be aware of its limitations until you encounter them.
 
-**🔄 Flows** (UI integration coming soon!): Design and run workflows with seamless automation.
-  - Create and run workflows with trigger-based automation:  
-    - Manual triggers or events from data, auth, media, or server actions.  
-    - HTTP triggers for external integrations.  
-  - Define tasks in sequence, parallel, or loops, with conditional execution.  
-  - Use reusable sub-workflows to organize complex processes.  
-  - Leverage OpenAPI specifications for API-based tasks.
+**The solution:** A backend system that only assumes and implements primitive details, integrates into multiple environments, and adheres to industry standards.
+
+## Features
+* ⚡ Instant backend with full REST API:
+  * **Data**: Define, query, and control your data with ease.
+  * **Auth**: Easily implement reliable authentication strategies.
+  * **Media**: Effortlessly manage and serve all your media files.
+  * **Flows**: Design and run workflows with seamless automation. (UI integration coming soon!)
+* 🌐 Built on Web Standards for maximum compatibility
+* 🏃‍♂️ Multiple run modes
+  * standalone using the CLI
+  * using a JavaScript runtime (Node, Bun, workerd)
+  * using a React framework (Astro, Remix, Next.js)
+* 📦 Official API and React SDK with type-safety
+* ⚛️ React elements for auto-configured authentication and media components
+
+## Structure
+The package is mainly split into 4 parts, each serving a specific purpose:
+
+| Import                      | Purpose                                              |
+|-----------------------------|------------------------------------------------------|
+| `bknd`<br/>`bknd/adapter/*` | Backend including APIs and adapters                  |
+| `bknd/ui`                   | Admin UI components for react frameworks             |
+| `bknd/client`               | TypeScript SDK and React hooks for the API endpoints |
+| `bknd/elements`             | React components for authentication and media        |
+
+
+### The backend (`bknd`)
+Serve the backend as an API for any JS runtime or framework. The latter is especially handy, as it allows you to deploy your frontend and backend bundled together. Furthermore it allows adding additional logic in a way you're already familar with. Just add another route and you're good to go.
+
+Here is an example of serving the API using node:
+```js index.js
+import { serve } from "bknd/adapter/node"
+serve();
+```
+
+### Integrated admin UI (`bknd/ui`)
+The admin UI allows to manage your data including full configuration of your backend using a graphical user interface. Using `vite`, your admin route looks like this:
+```tsx
+import { Admin } from "bknd/ui"
+import "bknd/dist/styles.css";
+
+export default function AdminPage() {
+   return <Admin />
+}
+```
+
+### Using the REST API or TypeScript SDK (`bknd/client`)
+If you're not using a JavaScript environment, you can still access any endpoint using the REST API:
+```bash
+curl -XGET <your-endpoint>/api/data/<entity>
+{
+  "data": [
+    { "id": 1, ... },
+    { "id": 2, ... }
+  ],
+  "meta": { /* ... */ }
+}
+```
+
+In a JavaScript environment, you can use the TypeScript SDK with type-safety. The above example would look like this:
+```ts
+import { Api } from "bknd/client";
+
+const api = new Api({ host: "<endpoint>" });
+const { data } = await api.data.readMany("<entity>");
+```
+
+If you're using React, there are 2 hooks exposed (`useApi`, `useEntity`), as well as an `swr` wrapper around each (`useApiQuery`, `useEntityQuery`). The `swr` wrapped hooks automatically handled query invalidation:
+
+```tsx
+import { useState } from "react";
+import { useEntityQuery } from "bknd/client";
+
+export default function App() {
+   const { data } = useEntityQuery("todos");   
+   return <ul>
+      {data?.map(todo => (
+         <li key={todo.id}>{todo.name}</li>
+      ))}
+   </ul>
+}
+```
+
+### React elements (`bknd/elements`)
+You don't have to figure out API details to include media uploads to your app. For an user avatar upload, this is all you need:
+```tsx
+import { Media } from "bknd/elements"
+import "bknd/dist/main.css"
+
+export function UserAvatar() {
+   return <Media.Dropzone
+     entity={{ name: "users", id: 1, field: "avatar" }}
+     maxItems={1}
+     overwrite
+   />
+}
+```
+The import path also exports components for login and registration forms which are automatically pointed to the `bknd` defaults.
 
 
 ## 🚀 Quick start

app/__test__/Module.spec.ts

@@ -1,38 +0,0 @@
-import { describe, expect, test } from "bun:test";
-import { type TSchema, Type, stripMark } from "../src/core/utils";
-import { Module } from "../src/modules/Module";
-
-function createModule<Schema extends TSchema>(schema: Schema) {
-   class TestModule extends Module<typeof schema> {
-      getSchema() {
-         return schema;
-      }
-      toJSON() {
-         return this.config;
-      }
-      useForceParse() {
-         return true;
-      }
-   }
-
-   return TestModule;
-}
-
-describe("Module", async () => {
-   test("basic", async () => {});
-
-   test("listener", async () => {
-      let result: any;
-
-      const module = createModule(Type.Object({ a: Type.String() }));
-      const m = new module({ a: "test" });
-
-      await m.schema().set({ a: "test2" });
-      m.setListener(async (c) => {
-         await new Promise((r) => setTimeout(r, 10));
-         result = stripMark(c);
-      });
-      await m.schema().set({ a: "test3" });
-      expect(result).toEqual({ a: "test3" });
-   });
-});

app/__test__/api/Api.spec.ts

@@ -0,0 +1,71 @@
+import { describe, expect, it } from "bun:test";
+import { sign } from "hono/jwt";
+import { Api } from "../../src/Api";
+
+describe("Api", async () => {
+   it("should construct without options", () => {
+      const api = new Api();
+      expect(api.baseUrl).toBe("http://localhost");
+      expect(api.isAuthVerified()).toBe(false);
+   });
+
+   it("should ignore force verify if no claims given", () => {
+      const api = new Api({ verified: true });
+      expect(api.baseUrl).toBe("http://localhost");
+      expect(api.isAuthVerified()).toBe(false);
+   });
+
+   it("should construct from request (token)", async () => {
+      const token = await sign({ foo: "bar" }, "1234");
+      const request = new Request("http://example.com/test", {
+         headers: {
+            Authorization: `Bearer ${token}`
+         }
+      });
+      const api = new Api({ request });
+      expect(api.isAuthVerified()).toBe(false);
+
+      const params = api.getParams();
+      expect(params.token).toBe(token);
+      expect(params.token_transport).toBe("header");
+      expect(params.host).toBe("http://example.com");
+   });
+
+   it("should construct from request (cookie)", async () => {
+      const token = await sign({ foo: "bar" }, "1234");
+      const request = new Request("http://example.com/test", {
+         headers: {
+            Cookie: `auth=${token}`
+         }
+      });
+      const api = new Api({ request });
+      expect(api.isAuthVerified()).toBe(false);
+
+      const params = api.getParams();
+      console.log(params);
+      expect(params.token).toBe(token);
+      expect(params.token_transport).toBe("cookie");
+      expect(params.host).toBe("http://example.com");
+   });
+
+   it("should construct from token", async () => {
+      const token = await sign({ foo: "bar" }, "1234");
+      const api = new Api({ token });
+      expect(api.isAuthVerified()).toBe(false);
+
+      const params = api.getParams();
+      expect(params.token).toBe(token);
+      expect(params.token_transport).toBe("header");
+      expect(params.host).toBe("http://localhost");
+   });
+
+   it("should prefer host when request is given", async () => {
+      const request = new Request("http://example.com/test");
+      const api = new Api({ request, host: "http://another.com" });
+
+      const params = api.getParams();
+      expect(params.token).toBeUndefined();
+      expect(params.token_transport).toBe("header");
+      expect(params.host).toBe("http://another.com");
+   });
+});

app/__test__/api/DataApi.spec.ts

@@ -0,0 +1,70 @@
+import { afterAll, beforeAll, describe, expect, it } from "bun:test";
+import { Guard } from "../../src/auth";
+import { parse } from "../../src/core/utils";
+import { DataApi } from "../../src/data/api/DataApi";
+import { DataController } from "../../src/data/api/DataController";
+import { dataConfigSchema } from "../../src/data/data-schema";
+import * as proto from "../../src/data/prototype";
+import { disableConsoleLog, enableConsoleLog, schemaToEm } from "../helper";
+
+beforeAll(disableConsoleLog);
+afterAll(enableConsoleLog);
+
+const dataConfig = parse(dataConfigSchema, {});
+describe("DataApi", () => {
+   it("should switch to post for long url reads", async () => {
+      const api = new DataApi();
+
+      const get = api.readMany("a".repeat(300), { select: ["id", "name"] });
+      expect(get.request.method).toBe("GET");
+      expect(new URL(get.request.url).pathname).toBe(`/api/data/${"a".repeat(300)}`);
+
+      const post = api.readMany("a".repeat(1000), { select: ["id", "name"] });
+      expect(post.request.method).toBe("POST");
+      expect(new URL(post.request.url).pathname).toBe(`/api/data/${"a".repeat(1000)}/query`);
+   });
+
+   it("returns result", async () => {
+      const schema = proto.em({
+         posts: proto.entity("posts", { title: proto.text() })
+      });
+      const em = schemaToEm(schema);
+      await em.schema().sync({ force: true });
+
+      const payload = [{ title: "foo" }, { title: "bar" }, { title: "baz" }];
+      await em.mutator("posts").insertMany(payload);
+
+      const ctx: any = { em, guard: new Guard() };
+      const controller = new DataController(ctx, dataConfig);
+      const app = controller.getController();
+
+      {
+         const res = (await app.request("/posts")) as Response;
+         const { data } = await res.json();
+         expect(data.length).toEqual(3);
+      }
+
+      // @ts-ignore tests
+      const api = new DataApi({ basepath: "/", queryLengthLimit: 50 });
+      // @ts-ignore protected
+      api.fetcher = app.request as typeof fetch;
+      {
+         const req = api.readMany("posts", { select: ["title"] });
+         expect(req.request.method).toBe("GET");
+         const res = await req;
+         expect(res.data).toEqual(payload);
+      }
+
+      {
+         const req = api.readMany("posts", {
+            select: ["title"],
+            limit: 100000,
+            offset: 0,
+            sort: "id"
+         });
+         expect(req.request.method).toBe("POST");
+         const res = await req;
+         expect(res.data).toEqual(payload);
+      }
+   });
+});

app/__test__/api/ModuleApi.spec.ts

@@ -27,8 +27,7 @@ describe("ModuleApi", () => {
 
    it("fetches endpoint", async () => {
       const app = new Hono().get("/endpoint", (c) => c.json({ foo: "bar" }));
-      const api = new Api({ host });
-      api.fetcher = app.request as typeof fetch;
+      const api = new Api({ host }, app.request as typeof fetch);
 
       const res = await api.get("/endpoint");
       expect(res.res.ok).toEqual(true);
@@ -39,8 +38,7 @@ describe("ModuleApi", () => {
 
    it("has accessible request", async () => {
       const app = new Hono().get("/endpoint", (c) => c.json({ foo: "bar" }));
-      const api = new Api({ host });
-      api.fetcher = app.request as typeof fetch;
+      const api = new Api({ host }, app.request as typeof fetch);
 
       const promise = api.get("/endpoint");
       expect(promise.request).toBeDefined();

app/__test__/app/repro.spec.ts

@@ -0,0 +1,73 @@
+import { describe, expect, test } from "bun:test";
+import { createApp, registries } from "../../src";
+import * as proto from "../../src/data/prototype";
+import { StorageLocalAdapter } from "../../src/media/storage/adapters/StorageLocalAdapter";
+
+describe("repros", async () => {
+   /**
+    * steps:
+    * 1. enable media
+    * 2. create 'test' entity
+    * 3. add media to 'test'
+    *
+    * There was an issue that AppData had old configs because of system entity "media"
+    */
+   test("registers media entity correctly to relate to it", async () => {
+      registries.media.register("local", StorageLocalAdapter);
+      const app = createApp();
+      await app.build();
+
+      {
+         // 1. enable media
+         const [, config] = await app.module.media.schema().patch("", {
+            enabled: true,
+            adapter: {
+               type: "local",
+               config: {
+                  path: "./"
+               }
+            }
+         });
+
+         expect(config.enabled).toBe(true);
+      }
+
+      {
+         // 2. create 'test' entity
+         await app.module.data.schema().patch(
+            "entities.test",
+            proto
+               .entity("test", {
+                  content: proto.text()
+               })
+               .toJSON()
+         );
+         expect(app.em.entities.map((e) => e.name)).toContain("test");
+      }
+
+      {
+         await app.module.data.schema().patch("entities.test.fields.files", {
+            type: "media",
+            config: {
+               required: false,
+               fillable: ["update"],
+               hidden: false,
+               mime_types: [],
+               virtual: true,
+               entity: "test"
+            }
+         });
+
+         expect(
+            app.module.data.schema().patch("relations.000", {
+               type: "poly",
+               source: "test",
+               target: "media",
+               config: { mappedBy: "files" }
+            })
+         ).resolves.toBeDefined();
+      }
+
+      expect(app.em.entities.map((e) => e.name)).toEqual(["media", "test"]);
+   });
+});

app/__test__/auth/middleware.spec.ts

@@ -0,0 +1 @@
+import { describe, expect, it } from "bun:test";

app/__test__/core/EventManager.spec.ts

@@ -1,8 +1,12 @@
-import { describe, expect, test } from "bun:test";
-import { Event, EventManager, NoParamEvent } from "../../src/core/events";
+import { afterAll, beforeAll, describe, expect, mock, test } from "bun:test";
+import { Event, EventManager, InvalidEventReturn, NoParamEvent } from "../../src/core/events";
+import { disableConsoleLog, enableConsoleLog } from "../helper";
+
+beforeAll(disableConsoleLog);
+afterAll(enableConsoleLog);
 
 class SpecialEvent extends Event<{ foo: string }> {
-   static slug = "special-event";
+   static override slug = "special-event";
 
    isBar() {
       return this.params.foo === "bar";
@@ -10,37 +14,139 @@ class SpecialEvent extends Event<{ foo: string }> {
 }
 
 class InformationalEvent extends NoParamEvent {
-   static slug = "informational-event";
+   static override slug = "informational-event";
+}
+
+class ReturnEvent extends Event<{ foo: string }, string> {
+   static override slug = "return-event";
+
+   override validate(value: string) {
+      if (typeof value !== "string") {
+         throw new InvalidEventReturn("string", typeof value);
+      }
+
+      return this.clone({
+         foo: [this.params.foo, value].join("-")
+      });
+   }
 }
 
 describe("EventManager", async () => {
-   test("test", async () => {
+   test("executes", async () => {
+      const call = mock(() => null);
+      const delayed = mock(() => null);
+
       const emgr = new EventManager();
       emgr.registerEvents([SpecialEvent, InformationalEvent]);
 
+      expect(emgr.eventExists("special-event")).toBe(true);
+      expect(emgr.eventExists("informational-event")).toBe(true);
+      expect(emgr.eventExists("unknown-event")).toBe(false);
+
       emgr.onEvent(
          SpecialEvent,
          async (event, name) => {
-            console.log("Event: ", name, event.params.foo, event.isBar());
-            console.log("wait...");
-
-            await new Promise((resolve) => setTimeout(resolve, 100));
-            console.log("done waiting");
+            expect(name).toBe("special-event");
+            expect(event.isBar()).toBe(true);
+            call();
+            await new Promise((resolve) => setTimeout(resolve, 50));
+            delayed();
          },
          "sync"
       );
 
+      // don't allow unknown
+      expect(() => emgr.on("unknown", () => void 0)).toThrow();
+
       emgr.onEvent(InformationalEvent, async (event, name) => {
-         console.log("Event: ", name, event.params);
+         call();
+         expect(name).toBe("informational-event");
       });
 
       await emgr.emit(new SpecialEvent({ foo: "bar" }));
-      console.log("done");
+      await emgr.emit(new InformationalEvent());
 
       // expect construct signatures to not cause ts errors
       new SpecialEvent({ foo: "bar" });
       new InformationalEvent();
 
-      expect(true).toBe(true);
+      expect(call).toHaveBeenCalledTimes(2);
+      expect(delayed).toHaveBeenCalled();
+   });
+
+   test("custom async executor", async () => {
+      const call = mock(() => null);
+      const asyncExecutor = (p: Promise<any>[]) => {
+         call();
+         return Promise.all(p);
+      };
+      const emgr = new EventManager(
+         { InformationalEvent },
+         {
+            asyncExecutor
+         }
+      );
+
+      emgr.onEvent(InformationalEvent, async () => {});
+      await emgr.emit(new InformationalEvent());
+      expect(call).toHaveBeenCalled();
+   });
+
+   test("piping", async () => {
+      const onInvalidReturn = mock(() => null);
+      const asyncEventCallback = mock(() => null);
+      const emgr = new EventManager(
+         { ReturnEvent, InformationalEvent },
+         {
+            onInvalidReturn
+         }
+      );
+
+      // @ts-expect-error InformationalEvent has no return value
+      emgr.onEvent(InformationalEvent, async () => {
+         asyncEventCallback();
+         return 1;
+      });
+
+      emgr.onEvent(ReturnEvent, async () => "1", "sync");
+      emgr.onEvent(ReturnEvent, async () => "0", "sync");
+
+      // @ts-expect-error must be string
+      emgr.onEvent(ReturnEvent, async () => 0, "sync");
+
+      // return is not required
+      emgr.onEvent(ReturnEvent, async () => {}, "sync");
+
+      // was "async", will not return
+      const e1 = await emgr.emit(new InformationalEvent());
+      expect(e1.returned).toBe(false);
+
+      const e2 = await emgr.emit(new ReturnEvent({ foo: "bar" }));
+      expect(e2.returned).toBe(true);
+      expect(e2.params.foo).toBe("bar-1-0");
+      expect(onInvalidReturn).toHaveBeenCalled();
+      expect(asyncEventCallback).toHaveBeenCalled();
+   });
+
+   test("once", async () => {
+      const call = mock(() => null);
+      const emgr = new EventManager({ InformationalEvent });
+
+      emgr.onEvent(
+         InformationalEvent,
+         async (event, slug) => {
+            expect(event).toBeInstanceOf(InformationalEvent);
+            expect(slug).toBe("informational-event");
+            call();
+         },
+         { mode: "sync", once: true }
+      );
+
+      expect(emgr.getListeners().length).toBe(1);
+      await emgr.emit(new InformationalEvent());
+      expect(emgr.getListeners().length).toBe(0);
+      await emgr.emit(new InformationalEvent());
+      expect(emgr.getListeners().length).toBe(0);
+      expect(call).toHaveBeenCalledTimes(1);
    });
 });

app/__test__/core/crypto.spec.ts

@@ -1,14 +1,16 @@
-import { describe, test } from "bun:test";
+import { describe, expect, test } from "bun:test";
 import { checksum, hash } from "../../src/core/utils";
 
 describe("crypto", async () => {
    test("sha256", async () => {
-      console.log(await hash.sha256("test"));
+      expect(await hash.sha256("test")).toBe(
+         "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
+      );
    });
    test("sha1", async () => {
-      console.log(await hash.sha1("test"));
+      expect(await hash.sha1("test")).toBe("a94a8fe5ccb19ba61c4c0873d391e987982fbbd3");
    });
    test("checksum", async () => {
-      console.log(checksum("hello world"));
+      expect(await checksum("hello world")).toBe("2aae6c35c94fcfb415dbe95f408b9ce91ee846ed");
    });
 });

app/__test__/core/utils.spec.ts

@@ -1,7 +1,6 @@
 import { describe, expect, test } from "bun:test";
 import { Perf } from "../../src/core/utils";
-import * as reqres from "../../src/core/utils/reqres";
-import * as strings from "../../src/core/utils/strings";
+import * as utils from "../../src/core/utils";
 
 async function wait(ms: number) {
    return new Promise((resolve) => {
@@ -13,7 +12,7 @@ describe("Core Utils", async () => {
    describe("[core] strings", async () => {
       test("objectToKeyValueArray", async () => {
          const obj = { a: 1, b: 2, c: 3 };
-         const result = strings.objectToKeyValueArray(obj);
+         const result = utils.objectToKeyValueArray(obj);
          expect(result).toEqual([
             { key: "a", value: 1 },
             { key: "b", value: 2 },
@@ -22,24 +21,24 @@ describe("Core Utils", async () => {
       });
 
       test("snakeToPascalWithSpaces", async () => {
-         const result = strings.snakeToPascalWithSpaces("snake_to_pascal");
+         const result = utils.snakeToPascalWithSpaces("snake_to_pascal");
          expect(result).toBe("Snake To Pascal");
       });
 
       test("randomString", async () => {
-         const result = strings.randomString(10);
+         const result = utils.randomString(10);
          expect(result).toHaveLength(10);
       });
 
       test("pascalToKebab", async () => {
-         const result = strings.pascalToKebab("PascalCase");
+         const result = utils.pascalToKebab("PascalCase");
          expect(result).toBe("pascal-case");
       });
 
       test("replaceSimplePlaceholders", async () => {
          const str = "Hello, {$name}!";
          const vars = { name: "John" };
-         const result = strings.replaceSimplePlaceholders(str, vars);
+         const result = utils.replaceSimplePlaceholders(str, vars);
          expect(result).toBe("Hello, John!");
       });
    });
@@ -49,7 +48,7 @@ describe("Core Utils", async () => {
          const headers = new Headers();
          headers.append("Content-Type", "application/json");
          headers.append("Authorization", "Bearer 123");
-         const obj = reqres.headersToObject(headers);
+         const obj = utils.headersToObject(headers);
          expect(obj).toEqual({
             "content-type": "application/json",
             authorization: "Bearer 123"
@@ -59,21 +58,21 @@ describe("Core Utils", async () => {
       test("replaceUrlParam", () => {
          const url = "/api/:id/:name";
          const params = { id: "123", name: "test" };
-         const result = reqres.replaceUrlParam(url, params);
+         const result = utils.replaceUrlParam(url, params);
          expect(result).toBe("/api/123/test");
       });
 
       test("encode", () => {
          const obj = { id: "123", name: "test" };
-         const result = reqres.encodeSearch(obj);
+         const result = utils.encodeSearch(obj);
          expect(result).toBe("id=123&name=test");
 
          const obj2 = { id: "123", name: ["test1", "test2"] };
-         const result2 = reqres.encodeSearch(obj2);
+         const result2 = utils.encodeSearch(obj2);
          expect(result2).toBe("id=123&name=test1&name=test2");
 
          const obj3 = { id: "123", name: { test: "test" } };
-         const result3 = reqres.encodeSearch(obj3, { encode: true });
+         const result3 = utils.encodeSearch(obj3, { encode: true });
          expect(result3).toBe("id=123&name=%7B%22test%22%3A%22test%22%7D");
       });
    });
@@ -108,4 +107,91 @@ describe("Core Utils", async () => {
          expect(count).toBe(2);
       });
    });
+
+   describe("objects", () => {
+      test("omitKeys", () => {
+         const objects = [
+            [{ a: 1, b: 2, c: 3 }, ["a"], { b: 2, c: 3 }],
+            [{ a: 1, b: 2, c: 3 }, ["b"], { a: 1, c: 3 }],
+            [{ a: 1, b: 2, c: 3 }, ["c"], { a: 1, b: 2 }],
+            [{ a: 1, b: 2, c: 3 }, ["a", "b"], { c: 3 }],
+            [{ a: 1, b: 2, c: 3 }, ["a", "b", "c"], {}]
+         ] as [object, string[], object][];
+
+         for (const [obj, keys, expected] of objects) {
+            const result = utils.omitKeys(obj, keys as any);
+            expect(result).toEqual(expected);
+         }
+      });
+
+      test("isEqual", () => {
+         const objects = [
+            [1, 1, true],
+            [1, "1", false],
+            [1, 2, false],
+            ["1", "1", true],
+            ["1", "2", false],
+            [true, true, true],
+            [true, false, false],
+            [false, false, true],
+            [1, NaN, false],
+            [NaN, NaN, true],
+            [null, null, true],
+            [null, undefined, false],
+            [undefined, undefined, true],
+            [new Map([["a", 1]]), new Map([["a", 1]]), true],
+            [new Map([["a", 1]]), new Map([["a", 2]]), false],
+            [new Map([["a", 1]]), new Map([["b", 1]]), false],
+            [
+               new Map([["a", 1]]),
+               new Map([
+                  ["a", 1],
+                  ["b", 2]
+               ]),
+               false
+            ],
+            [{ a: 1 }, { a: 1 }, true],
+            [{ a: 1 }, { a: 2 }, false],
+            [{ a: 1 }, { b: 1 }, false],
+            [{ a: "1" }, { a: "1" }, true],
+            [{ a: "1" }, { a: "2" }, false],
+            [{ a: "1" }, { b: "1" }, false],
+            [{ a: 1 }, { a: 1, b: 2 }, false],
+            [{ a: [1, 2, 3] }, { a: [1, 2, 3] }, true],
+            [{ a: [1, 2, 3] }, { a: [1, 2, 4] }, false],
+            [{ a: [1, 2, 3] }, { a: [1, 2, 3, 4] }, false],
+            [{ a: { b: 1 } }, { a: { b: 1 } }, true],
+            [{ a: { b: 1 } }, { a: { b: 2 } }, false],
+            [{ a: { b: 1 } }, { a: { c: 1 } }, false],
+            [{ a: { b: 1 } }, { a: { b: 1, c: 2 } }, false],
+            [[1, 2, 3], [1, 2, 3], true],
+            [[1, 2, 3], [1, 2, 4], false],
+            [[1, 2, 3], [1, 2, 3, 4], false],
+            [[{ a: 1 }], [{ a: 1 }], true],
+            [[{ a: 1 }], [{ a: 2 }], false],
+            [[{ a: 1 }], [{ b: 1 }], false]
+         ] as [any, any, boolean][];
+
+         for (const [a, b, expected] of objects) {
+            const result = utils.isEqual(a, b);
+            expect(result).toEqual(expected);
+         }
+      });
+
+      test("getPath", () => {
+         const tests = [
+            [{ a: 1, b: 2, c: 3 }, "a", 1],
+            [{ a: 1, b: 2, c: 3 }, "b", 2],
+            [{ a: { b: 1 } }, "a.b", 1],
+            [{ a: { b: 1 } }, "a.b.c", null, null],
+            [{ a: { b: 1 } }, "a.b.c", 1, 1],
+            [[[1]], "0.0", 1]
+         ] as [object, string, any, any][];
+
+         for (const [obj, path, expected, defaultValue] of tests) {
+            const result = utils.getPath(obj, path, defaultValue);
+            expect(result).toEqual(expected);
+         }
+      });
+   });
 });

app/__test__/data/data-query-impl.spec.ts

@@ -1,16 +1,21 @@
 import { describe, expect, test } from "bun:test";
-import type { QueryObject } from "ufo";
-import { WhereBuilder, type WhereQuery } from "../../src/data/entities/query/WhereBuilder";
+import { Value, _jsonp } from "../../src/core/utils";
+import { type RepoQuery, WhereBuilder, type WhereQuery, querySchema } from "../../src/data";
+import type { RepoQueryIn } from "../../src/data/server/data-query-impl";
 import { getDummyConnection } from "./helper";
 
-const t = "t";
+const decode = (input: RepoQueryIn, expected: RepoQuery) => {
+   const result = Value.Decode(querySchema, input);
+   expect(result).toEqual(expected);
+};
+
 describe("data-query-impl", () => {
    function qb() {
       const c = getDummyConnection();
       const kysely = c.dummyConnection.kysely;
-      return kysely.selectFrom(t).selectAll();
+      return kysely.selectFrom("t").selectAll();
    }
-   function compile(q: QueryObject) {
+   function compile(q: WhereQuery) {
       const { sql, parameters } = WhereBuilder.addClause(qb(), q).compile();
       return { sql, parameters };
    }
@@ -89,4 +94,64 @@ describe("data-query-impl", () => {
          expect(keys).toEqual(expectedKeys);
       }
    });
+
+   test("with", () => {
+      decode({ with: ["posts"] }, { with: { posts: {} } });
+      decode({ with: { posts: {} } }, { with: { posts: {} } });
+      decode({ with: { posts: { limit: 1 } } }, { with: { posts: { limit: 1 } } });
+      decode(
+         {
+            with: {
+               posts: {
+                  with: {
+                     images: {
+                        select: ["id"]
+                     }
+                  }
+               }
+            }
+         },
+         {
+            with: {
+               posts: {
+                  with: {
+                     images: {
+                        select: ["id"]
+                     }
+                  }
+               }
+            }
+         }
+      );
+
+      // over http
+      {
+         const output = { with: { images: {} } };
+         decode({ with: "images" }, output);
+         decode({ with: '["images"]' }, output);
+         decode({ with: ["images"] }, output);
+         decode({ with: { images: {} } }, output);
+      }
+
+      {
+         const output = { with: { images: {}, comments: {} } };
+         decode({ with: "images,comments" }, output);
+         decode({ with: ["images", "comments"] }, output);
+         decode({ with: '["images", "comments"]' }, output);
+         decode({ with: { images: {}, comments: {} } }, output);
+      }
+   });
+});
+
+describe("data-query-impl: Typebox", () => {
+   test("sort", async () => {
+      const _dflt = { sort: { by: "id", dir: "asc" } };
+
+      decode({ sort: "" }, _dflt);
+      decode({ sort: "name" }, { sort: { by: "name", dir: "asc" } });
+      decode({ sort: "-name" }, { sort: { by: "name", dir: "desc" } });
+      decode({ sort: "-posts.name" }, { sort: { by: "posts.name", dir: "desc" } });
+      decode({ sort: "-1name" }, _dflt);
+      decode({ sort: { by: "name", dir: "desc" } }, { sort: { by: "name", dir: "desc" } });
+   });
 });

app/__test__/data/data.test.ts

@@ -18,7 +18,7 @@ describe("some tests", async () => {
 
    const users = new Entity("users", [
       new TextField("username", { required: true, default_value: "nobody" }),
-      new TextField("email", { max_length: 3 })
+      new TextField("email", { maxLength: 3 })
    ]);
 
    const posts = new Entity("posts", [

app/__test__/data/relations.test.ts

@@ -106,7 +106,6 @@ describe("Relations", async () => {
       expect(postAuthorRel?.other(posts).entity).toBe(users);
 
       const kysely = em.connection.kysely;
-      const jsonFrom = (e) => e;
       /**
        * Relation Helper
        */
@@ -119,14 +118,11 @@ describe("Relations", async () => {
        - select: users.*
        - cardinality: 1
        */
-      const selectPostsFromUsers = postAuthorRel.buildWith(
-         users,
-         kysely.selectFrom(users.name),
-         jsonFrom,
-         "posts"
-      );
+      const selectPostsFromUsers = kysely
+         .selectFrom(users.name)
+         .select((eb) => postAuthorRel.buildWith(users, "posts")(eb).as("posts"));
       expect(selectPostsFromUsers.compile().sql).toBe(
-         'select (select "posts"."id" as "id", "posts"."title" as "title", "posts"."author_id" as "author_id" from "posts" as "posts" where "posts"."author_id" = "users"."id" limit ?) as "posts" from "users"'
+         'select (select from "posts" as "posts" where "posts"."author_id" = "users"."id") as "posts" from "users"'
       );
       expect(postAuthorRel!.getField()).toBeInstanceOf(RelationField);
       const userObj = { id: 1, username: "test" };
@@ -141,15 +137,12 @@ describe("Relations", async () => {
        - select: posts.*
        - cardinality:
        */
-      const selectUsersFromPosts = postAuthorRel.buildWith(
-         posts,
-         kysely.selectFrom(posts.name),
-         jsonFrom,
-         "author"
-      );
+      const selectUsersFromPosts = kysely
+         .selectFrom(posts.name)
+         .select((eb) => postAuthorRel.buildWith(posts, "author")(eb).as("author"));
 
       expect(selectUsersFromPosts.compile().sql).toBe(
-         'select (select "author"."id" as "id", "author"."username" as "username" from "users" as "author" where "author"."id" = "posts"."author_id" limit ?) as "author" from "posts"'
+         'select (select from "users" as "author" where "author"."id" = "posts"."author_id" limit ?) as "author" from "posts"'
       );
       expect(postAuthorRel.getField()).toBeInstanceOf(RelationField);
       const postObj = { id: 1, title: "test" };
@@ -315,20 +308,16 @@ describe("Relations", async () => {
        - select: users.*
        - cardinality: 1
        */
-      const selectCategoriesFromPosts = postCategoriesRel.buildWith(
-         posts,
-         kysely.selectFrom(posts.name),
-         jsonFrom
-      );
+      const selectCategoriesFromPosts = kysely
+         .selectFrom(posts.name)
+         .select((eb) => postCategoriesRel.buildWith(posts)(eb).as("categories"));
       expect(selectCategoriesFromPosts.compile().sql).toBe(
          'select (select "categories"."id" as "id", "categories"."label" as "label" from "categories" inner join "posts_categories" on "categories"."id" = "posts_categories"."categories_id" where "posts"."id" = "posts_categories"."posts_id" limit ?) as "categories" from "posts"'
       );
 
-      const selectPostsFromCategories = postCategoriesRel.buildWith(
-         categories,
-         kysely.selectFrom(categories.name),
-         jsonFrom
-      );
+      const selectPostsFromCategories = kysely
+         .selectFrom(categories.name)
+         .select((eb) => postCategoriesRel.buildWith(categories)(eb).as("posts"));
       expect(selectPostsFromCategories.compile().sql).toBe(
          'select (select "posts"."id" as "id", "posts"."title" as "title" from "posts" inner join "posts_categories" on "posts"."id" = "posts_categories"."posts_id" where "categories"."id" = "posts_categories"."categories_id" limit ?) as "posts" from "categories"'
       );

app/__test__/data/specs/Mutator.spec.ts

@@ -1,4 +1,5 @@
 import { afterAll, describe, expect, test } from "bun:test";
+import type { EventManager } from "../../../src/core/events";
 import {
    Entity,
    EntityManager,
@@ -10,6 +11,7 @@ import {
    RelationMutator,
    TextField
 } from "../../../src/data";
+import * as proto from "../../../src/data/prototype";
 import { getDummyConnection } from "../helper";
 
 const { dummyConnection, afterAllCleanup } = getDummyConnection();
@@ -83,14 +85,12 @@ describe("[data] Mutator (ManyToOne)", async () => {
 
       // persisting reference should ...
       expect(
-         postRelMutator.persistReference(relations[0], "users", {
+         postRelMutator.persistReference(relations[0]!, "users", {
             $set: { id: userData.data.id }
          })
       ).resolves.toEqual(["users_id", userData.data.id]);
       // @todo: add what methods are allowed to relation, like $create should not be allowed for post<>users
 
-      process.exit(0);
-
       const userRelMutator = new RelationMutator(users, em);
       expect(userRelMutator.getRelationalKeys()).toEqual(["posts"]);
    });
@@ -99,7 +99,7 @@ describe("[data] Mutator (ManyToOne)", async () => {
       expect(
          em.mutator(posts).insertOne({
             title: "post1",
-            users_id: 1 // user does not exist yet
+            users_id: 100 // user does not exist yet
          })
       ).rejects.toThrow();
    });
@@ -299,4 +299,71 @@ describe("[data] Mutator (Events)", async () => {
       expect(events.has(MutatorEvents.MutatorDeleteBefore.slug)).toBeTrue();
       expect(events.has(MutatorEvents.MutatorDeleteAfter.slug)).toBeTrue();
    });
+
+   test("insertOne event return is respected", async () => {
+      const posts = proto.entity("posts", {
+         title: proto.text(),
+         views: proto.number()
+      });
+
+      const conn = getDummyConnection();
+      const em = new EntityManager([posts], conn.dummyConnection);
+      await em.schema().sync({ force: true });
+
+      const emgr = em.emgr as EventManager<any>;
+
+      emgr.onEvent(
+         // @ts-ignore
+         EntityManager.Events.MutatorInsertBefore,
+         async (event) => {
+            return {
+               ...event.params.data,
+               views: 2
+            };
+         },
+         "sync"
+      );
+
+      const mutator = em.mutator("posts");
+      const result = await mutator.insertOne({ title: "test", views: 1 });
+      expect(result.data).toEqual({
+         id: 1,
+         title: "test",
+         views: 2
+      });
+   });
+
+   test("updateOne event return is respected", async () => {
+      const posts = proto.entity("posts", {
+         title: proto.text(),
+         views: proto.number()
+      });
+
+      const conn = getDummyConnection();
+      const em = new EntityManager([posts], conn.dummyConnection);
+      await em.schema().sync({ force: true });
+
+      const emgr = em.emgr as EventManager<any>;
+
+      emgr.onEvent(
+         // @ts-ignore
+         EntityManager.Events.MutatorUpdateBefore,
+         async (event) => {
+            return {
+               ...event.params.data,
+               views: event.params.data.views + 1
+            };
+         },
+         "sync"
+      );
+
+      const mutator = em.mutator("posts");
+      const created = await mutator.insertOne({ title: "test", views: 1 });
+      const result = await mutator.updateOne(created.data.id, { views: 2 });
+      expect(result.data).toEqual({
+         id: 1,
+         title: "test",
+         views: 3
+      });
+   });
 });

app/__test__/data/specs/Repository.spec.ts

@@ -1,7 +1,6 @@
 import { afterAll, describe, expect, test } from "bun:test";
-// @ts-ignore
-import { Perf } from "@bknd/core/utils";
 import type { Kysely, Transaction } from "kysely";
+import { Perf } from "../../../src/core/utils";
 import {
    Entity,
    EntityManager,
@@ -24,7 +23,7 @@ async function sleep(ms: number) {
 }
 
 describe("[Repository]", async () => {
-   test("bulk", async () => {
+   test.skip("bulk", async () => {
       //const connection = dummyConnection;
       //const connection = getLocalLibsqlConnection();
       const credentials = null as any; // @todo: determine what to do here

app/__test__/data/specs/WithBuilder.spec.ts

@@ -1,4 +1,5 @@
 import { afterAll, describe, expect, test } from "bun:test";
+import { _jsonp } from "../../../src/core/utils";
 import {
    Entity,
    EntityManager,
@@ -8,19 +9,56 @@ import {
    TextField,
    WithBuilder
 } from "../../../src/data";
+import * as proto from "../../../src/data/prototype";
+import { compileQb, prettyPrintQb, schemaToEm } from "../../helper";
 import { getDummyConnection } from "../helper";
 
-const { dummyConnection, afterAllCleanup } = getDummyConnection();
-afterAll(afterAllCleanup);
+const { dummyConnection } = getDummyConnection();
 
 describe("[data] WithBuilder", async () => {
+   test("validate withs", async () => {
+      const schema = proto.em(
+         {
+            posts: proto.entity("posts", {}),
+            users: proto.entity("users", {}),
+            media: proto.entity("media", {})
+         },
+         ({ relation }, { posts, users, media }) => {
+            relation(posts).manyToOne(users);
+            relation(users).polyToOne(media, { mappedBy: "avatar" });
+         }
+      );
+      const em = schemaToEm(schema);
+
+      expect(WithBuilder.validateWiths(em, "posts", undefined)).toBe(0);
+      expect(WithBuilder.validateWiths(em, "posts", {})).toBe(0);
+      expect(WithBuilder.validateWiths(em, "posts", { users: {} })).toBe(1);
+      expect(
+         WithBuilder.validateWiths(em, "posts", {
+            users: {
+               with: { avatar: {} }
+            }
+         })
+      ).toBe(2);
+      expect(() => WithBuilder.validateWiths(em, "posts", { author: {} })).toThrow();
+      expect(() =>
+         WithBuilder.validateWiths(em, "posts", {
+            users: {
+               with: { glibberish: {} }
+            }
+         })
+      ).toThrow();
+   });
+
    test("missing relation", async () => {
       const users = new Entity("users", [new TextField("username")]);
       const em = new EntityManager([users], dummyConnection);
 
       expect(() =>
-         WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, ["posts"])
-      ).toThrow('Relation "posts" not found');
+         WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, {
+            posts: {}
+         })
+      ).toThrow('Relation "users<>posts" not found');
    });
 
    test("addClause: ManyToOne", async () => {
@@ -29,36 +67,39 @@ describe("[data] WithBuilder", async () => {
       const relations = [new ManyToOneRelation(posts, users, { mappedBy: "author" })];
       const em = new EntityManager([users, posts], dummyConnection, relations);
 
-      const qb = WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, [
-         "posts"
-      ]);
+      const qb = WithBuilder.addClause(em, em.connection.kysely.selectFrom("users"), users, {
+         posts: {}
+      });
 
       const res = qb.compile();
 
       expect(res.sql).toBe(
-         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'content\', "agg"."content", \'author_id\', "agg"."author_id")), \'[]\') from (select "posts"."id" as "id", "posts"."content" as "content", "posts"."author_id" as "author_id" from "posts" where "users"."id" = "posts"."author_id" limit ?) as agg) as "posts" from "users"'
+         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'content\', "agg"."content", \'author_id\', "agg"."author_id")), \'[]\') from (select "posts"."id" as "id", "posts"."content" as "content", "posts"."author_id" as "author_id" from "posts" as "posts" where "posts"."author_id" = "users"."id" order by "posts"."id" asc limit ? offset ?) as agg) as "posts" from "users"'
       );
-      expect(res.parameters).toEqual([5]);
+      expect(res.parameters).toEqual([10, 0]);
 
       const qb2 = WithBuilder.addClause(
          em,
          em.connection.kysely.selectFrom("posts"),
          posts, // @todo: try with "users", it gives output!
-         ["author"]
+         {
+            author: {}
+         }
       );
 
       const res2 = qb2.compile();
 
       expect(res2.sql).toBe(
-         'select (select json_object(\'id\', "obj"."id", \'username\', "obj"."username") from (select "users"."id" as "id", "users"."username" as "username" from "users" where "posts"."author_id" = "users"."id" limit ?) as obj) as "author" from "posts"'
+         'select (select json_object(\'id\', "obj"."id", \'username\', "obj"."username") from (select "users"."id" as "id", "users"."username" as "username" from "users" as "author" where "author"."id" = "posts"."author_id" order by "users"."id" asc limit ? offset ?) as obj) as "author" from "posts"'
       );
-      expect(res2.parameters).toEqual([1]);
+      expect(res2.parameters).toEqual([1, 0]);
    });
 
    test("test with empty join", async () => {
+      const em = new EntityManager([], dummyConnection);
       const qb = { qb: 1 } as any;
 
-      expect(WithBuilder.addClause(null as any, qb, null as any, [])).toBe(qb);
+      expect(WithBuilder.addClause(em, qb, null as any, {})).toBe(qb);
    });
 
    test("test manytomany", async () => {
@@ -89,7 +130,7 @@ describe("[data] WithBuilder", async () => {
 
       //console.log((await em.repository().findMany("posts_categories")).result);
 
-      const res = await em.repository(posts).findMany({ with: ["categories"] });
+      const res = await em.repository(posts).findMany({ with: { categories: {} } });
 
       expect(res.data).toEqual([
          {
@@ -107,7 +148,7 @@ describe("[data] WithBuilder", async () => {
          }
       ]);
 
-      const res2 = await em.repository(categories).findMany({ with: ["posts"] });
+      const res2 = await em.repository(categories).findMany({ with: { posts: {} } });
 
       //console.log(res2.sql, res2.data);
 
@@ -121,8 +162,8 @@ describe("[data] WithBuilder", async () => {
             id: 2,
             label: "beauty",
             posts: [
-               { id: 2, title: "beauty post" },
-               { id: 1, title: "fashion post" }
+               { id: 1, title: "fashion post" },
+               { id: 2, title: "beauty post" }
             ]
          },
          {
@@ -150,25 +191,25 @@ describe("[data] WithBuilder", async () => {
          em,
          em.connection.kysely.selectFrom("categories"),
          categories,
-         ["single"]
+         { single: {} }
       );
       const res = qb.compile();
       expect(res.sql).toBe(
-         'select (select json_object(\'id\', "obj"."id", \'path\', "obj"."path") from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" limit ?) as obj) as "single" from "categories"'
+         'select (select json_object(\'id\', "obj"."id", \'path\', "obj"."path") from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" order by "media"."id" asc limit ? offset ?) as obj) as "single" from "categories"'
       );
-      expect(res.parameters).toEqual(["categories.single", 1]);
+      expect(res.parameters).toEqual(["categories.single", 1, 0]);
 
       const qb2 = WithBuilder.addClause(
          em,
          em.connection.kysely.selectFrom("categories"),
          categories,
-         ["multiple"]
+         { multiple: {} }
       );
       const res2 = qb2.compile();
       expect(res2.sql).toBe(
-         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'path\', "agg"."path")), \'[]\') from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" limit ?) as agg) as "multiple" from "categories"'
+         'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'path\', "agg"."path")), \'[]\') from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "categories"."id" = "media"."entity_id" order by "media"."id" asc limit ? offset ?) as agg) as "multiple" from "categories"'
       );
-      expect(res2.parameters).toEqual(["categories.multiple", 5]);
+      expect(res2.parameters).toEqual(["categories.multiple", 10, 0]);
    });
 
    /*test("test manytoone", async () => {
@@ -192,4 +233,205 @@ describe("[data] WithBuilder", async () => {
       const res = await em.repository().findMany("posts", { join: ["author"] });
       console.log(res.sql, res.parameters, res.result);
    });*/
+
+   describe("recursive", () => {
+      test("compiles with singles", async () => {
+         const schema = proto.em(
+            {
+               posts: proto.entity("posts", {}),
+               users: proto.entity("users", {
+                  username: proto.text()
+               }),
+               media: proto.entity("media", {
+                  path: proto.text()
+               })
+            },
+            ({ relation }, { posts, users, media }) => {
+               relation(posts).manyToOne(users);
+               relation(users).polyToOne(media, { mappedBy: "avatar" });
+            }
+         );
+         const em = schemaToEm(schema);
+
+         const qb = WithBuilder.addClause(
+            em,
+            em.connection.kysely.selectFrom("posts"),
+            schema.entities.posts,
+            {
+               users: {
+                  limit: 5, // ignored
+                  select: ["id", "username"],
+                  sort: { by: "username", dir: "asc" },
+                  with: {
+                     avatar: {
+                        select: ["id", "path"],
+                        limit: 2 // ignored
+                     }
+                  }
+               }
+            }
+         );
+
+         //prettyPrintQb(qb);
+         expect(qb.compile().sql).toBe(
+            'select (select json_object(\'id\', "obj"."id", \'username\', "obj"."username", \'avatar\', "obj"."avatar") from (select "users"."id" as "id", "users"."username" as "username", (select json_object(\'id\', "obj"."id", \'path\', "obj"."path") from (select "media"."id" as "id", "media"."path" as "path" from "media" where "media"."reference" = ? and "users"."id" = "media"."entity_id" order by "media"."id" asc limit ? offset ?) as obj) as "avatar" from "users" as "users" where "users"."id" = "posts"."users_id" order by "users"."username" asc limit ? offset ?) as obj) as "users" from "posts"'
+         );
+         expect(qb.compile().parameters).toEqual(["users.avatar", 1, 0, 1, 0]);
+      });
+
+      test("compiles with many", async () => {
+         const schema = proto.em(
+            {
+               posts: proto.entity("posts", {}),
+               comments: proto.entity("comments", {}),
+               users: proto.entity("users", {
+                  username: proto.text()
+               }),
+               media: proto.entity("media", {
+                  path: proto.text()
+               })
+            },
+            ({ relation }, { posts, comments, users, media }) => {
+               relation(posts).manyToOne(users).polyToOne(media, { mappedBy: "images" });
+               relation(users).polyToOne(media, { mappedBy: "avatar" });
+               relation(comments).manyToOne(posts).manyToOne(users);
+            }
+         );
+         const em = schemaToEm(schema);
+
+         const qb = WithBuilder.addClause(
+            em,
+            em.connection.kysely.selectFrom("posts"),
+            schema.entities.posts,
+            {
+               comments: {
+                  limit: 12,
+                  with: {
+                     users: {
+                        select: ["username"]
+                     }
+                  }
+               }
+            }
+         );
+
+         expect(qb.compile().sql).toBe(
+            'select (select coalesce(json_group_array(json_object(\'id\', "agg"."id", \'posts_id\', "agg"."posts_id", \'users_id\', "agg"."users_id", \'users\', "agg"."users")), \'[]\') from (select "comments"."id" as "id", "comments"."posts_id" as "posts_id", "comments"."users_id" as "users_id", (select json_object(\'username\', "obj"."username") from (select "users"."username" as "username" from "users" as "users" where "users"."id" = "comments"."users_id" order by "users"."id" asc limit ? offset ?) as obj) as "users" from "comments" as "comments" where "comments"."posts_id" = "posts"."id" order by "comments"."id" asc limit ? offset ?) as agg) as "comments" from "posts"'
+         );
+         expect(qb.compile().parameters).toEqual([1, 0, 12, 0]);
+      });
+
+      test("returns correct result", async () => {
+         const schema = proto.em(
+            {
+               posts: proto.entity("posts", {
+                  title: proto.text()
+               }),
+               comments: proto.entity("comments", {
+                  content: proto.text()
+               }),
+               users: proto.entity("users", {
+                  username: proto.text()
+               }),
+               media: proto.entity("media", {
+                  path: proto.text()
+               })
+            },
+            ({ relation }, { posts, comments, users, media }) => {
+               relation(posts).manyToOne(users).polyToOne(media, { mappedBy: "images" });
+               relation(users).polyToOne(media, { mappedBy: "avatar" });
+               relation(comments).manyToOne(posts).manyToOne(users);
+            }
+         );
+         const em = schemaToEm(schema);
+         await em.schema().sync({ force: true });
+
+         // add data
+         await em.mutator("users").insertMany([{ username: "user1" }, { username: "user2" }]);
+         await em.mutator("posts").insertMany([
+            { title: "post1", users_id: 1 },
+            { title: "post2", users_id: 1 },
+            { title: "post3", users_id: 2 }
+         ]);
+         await em.mutator("comments").insertMany([
+            { content: "comment1", posts_id: 1, users_id: 1 },
+            { content: "comment1-1", posts_id: 1, users_id: 1 },
+            { content: "comment2", posts_id: 1, users_id: 2 },
+            { content: "comment3", posts_id: 2, users_id: 1 },
+            { content: "comment4", posts_id: 2, users_id: 2 },
+            { content: "comment5", posts_id: 3, users_id: 1 },
+            { content: "comment6", posts_id: 3, users_id: 2 }
+         ]);
+
+         const result = await em.repo("posts").findMany({
+            select: ["title"],
+            with: {
+               comments: {
+                  limit: 2,
+                  select: ["content"],
+                  with: {
+                     users: {
+                        select: ["username"]
+                     }
+                  }
+               }
+            }
+         });
+
+         expect(result.data).toEqual([
+            {
+               title: "post1",
+               comments: [
+                  {
+                     content: "comment1",
+                     users: {
+                        username: "user1"
+                     }
+                  },
+                  {
+                     content: "comment1-1",
+                     users: {
+                        username: "user1"
+                     }
+                  }
+               ]
+            },
+            {
+               title: "post2",
+               comments: [
+                  {
+                     content: "comment3",
+                     users: {
+                        username: "user1"
+                     }
+                  },
+                  {
+                     content: "comment4",
+                     users: {
+                        username: "user2"
+                     }
+                  }
+               ]
+            },
+            {
+               title: "post3",
+               comments: [
+                  {
+                     content: "comment5",
+                     users: {
+                        username: "user1"
+                     }
+                  },
+                  {
+                     content: "comment6",
+                     users: {
+                        username: "user2"
+                     }
+                  }
+               ]
+            }
+         ]);
+         //console.log(_jsonp(result.data));
+      });
+   });
 });

app/__test__/data/specs/fields/EnumField.spec.ts

@@ -13,10 +13,6 @@ describe("[data] EnumField", async () => {
       { options: options(["a", "b", "c"]) }
    );
 
-   test("yields if no options", async () => {
-      expect(() => new EnumField("test", { options: options([]) })).toThrow();
-   });
-
    test("yields if default value is not a valid option", async () => {
       expect(
          () => new EnumField("test", { options: options(["a", "b"]), default_value: "c" })

app/__test__/data/specs/fields/Field.spec.ts

@@ -15,11 +15,9 @@ describe("[data] Field", async () => {
 
    runBaseFieldTests(FieldSpec, { defaultValue: "test", schemaType: "text" });
 
-   test.only("default config", async () => {
-      const field = new FieldSpec("test");
+   test("default config", async () => {
       const config = Default(baseFieldConfigSchema, {});
       expect(stripMark(new FieldSpec("test").config)).toEqual(config);
-      console.log("config", new TextField("test", { required: true }).toJSON());
    });
 
    test("transformPersist (specific)", async () => {

app/__test__/data/specs/fields/JsonField.spec.ts

@@ -32,7 +32,7 @@ describe("[data] JsonField", async () => {
    });
 
    test("getValue", async () => {
-      expect(field.getValue({ test: 1 }, "form")).toBe('{"test":1}');
+      expect(field.getValue({ test: 1 }, "form")).toBe('{\n  "test": 1\n}');
       expect(field.getValue("string", "form")).toBe('"string"');
       expect(field.getValue(1, "form")).toBe("1");
 

app/__test__/data/specs/relations/EntityRelation.spec.ts

@@ -70,9 +70,9 @@ describe("[data] EntityRelation", async () => {
 
    it("required", async () => {
       const relation1 = new TestEntityRelation();
-      expect(relation1.config.required).toBe(false);
+      expect(relation1.required).toBe(false);
 
       const relation2 = new TestEntityRelation({ required: true });
-      expect(relation2.config.required).toBe(true);
+      expect(relation2.required).toBe(true);
    });
 });

app/__test__/flows/workflow-basic.test.ts

@@ -1,7 +1,7 @@
 // eslint-disable-next-line import/no-unresolved
 import { describe, expect, test } from "bun:test";
 import { isEqual } from "lodash-es";
-import { type Static, Type, _jsonp } from "../../src/core/utils";
+import { type Static, Type, _jsonp, withDisabledConsole } from "../../src/core/utils";
 import { Condition, ExecutionEvent, FetchTask, Flow, LogTask, Task } from "../../src/flows";
 
 /*beforeAll(disableConsoleLog);
@@ -232,7 +232,9 @@ describe("Flow tests", async () => {
       ).toEqual(["second", "fourth"]);
 
       const execution = back.createExecution();
-      expect(execution.start()).rejects.toThrow();
+      withDisabledConsole(async () => {
+         expect(execution.start()).rejects.toThrow();
+      });
    });
 
    test("Flow with back step: enough retries", async () => {

app/__test__/helper.ts

@@ -1,7 +1,9 @@
 import { unlink } from "node:fs/promises";
-import type { SqliteDatabase } from "kysely";
+import type { SelectQueryBuilder, SqliteDatabase } from "kysely";
 import Database from "libsql";
-import { SqliteLocalConnection } from "../src/data";
+import { format as sqlFormat } from "sql-formatter";
+import { type Connection, EntityManager, SqliteLocalConnection } from "../src/data";
+import type { em as protoEm } from "../src/data/prototype";
 
 export function getDummyDatabase(memory: boolean = true): {
    dummyDb: SqliteDatabase;
@@ -40,7 +42,7 @@ const _oldConsoles = {
    error: console.error
 };
 
-export function disableConsoleLog(severities: ConsoleSeverity[] = ["log"]) {
+export function disableConsoleLog(severities: ConsoleSeverity[] = ["log", "warn"]) {
    severities.forEach((severity) => {
       console[severity] = () => null;
    });
@@ -51,3 +53,18 @@ export function enableConsoleLog() {
       console[severity as ConsoleSeverity] = fn;
    });
 }
+
+export function compileQb(qb: SelectQueryBuilder<any, any, any>) {
+   const { sql, parameters } = qb.compile();
+   return { sql, parameters };
+}
+
+export function prettyPrintQb(qb: SelectQueryBuilder<any, any, any>) {
+   const { sql, parameters } = qb.compile();
+   console.log("$", sqlFormat(sql), "\n[params]", parameters);
+}
+
+export function schemaToEm(s: ReturnType<typeof protoEm>, conn?: Connection): EntityManager<any> {
+   const connection = conn ? conn : getDummyConnection().dummyConnection;
+   return new EntityManager(Object.values(s.entities), connection, s.relations, s.indices);
+}

app/__test__/integration/auth.integration.test.ts

@@ -0,0 +1,213 @@
+import { afterAll, beforeAll, describe, expect, it } from "bun:test";
+import { App, createApp } from "../../src";
+import type { AuthResponse } from "../../src/auth";
+import { randomString, secureRandomString, withDisabledConsole } from "../../src/core/utils";
+import { disableConsoleLog, enableConsoleLog } from "../helper";
+
+beforeAll(disableConsoleLog);
+afterAll(enableConsoleLog);
+
+const roles = {
+   sloppy: {
+      guest: {
+         permissions: [
+            "system.access.admin",
+            "system.schema.read",
+            "system.access.api",
+            "system.config.read",
+            "data.entity.read"
+         ],
+         is_default: true
+      },
+      admin: {
+         is_default: true,
+         implicit_allow: true
+      }
+   },
+   strict: {
+      guest: {
+         permissions: ["system.access.api", "system.config.read", "data.entity.read"],
+         is_default: true
+      },
+      admin: {
+         is_default: true,
+         implicit_allow: true
+      }
+   }
+};
+const configs = {
+   auth: {
+      enabled: true,
+      entity_name: "users",
+      jwt: {
+         secret: secureRandomString(20),
+         issuer: randomString(10)
+      },
+      roles: roles.strict,
+      guard: {
+         enabled: true
+      }
+   },
+   users: {
+      normal: {
+         email: "normal@bknd.io",
+         password: "12345678"
+      },
+      admin: {
+         email: "admin@bknd.io",
+         password: "12345678",
+         role: "admin"
+      }
+   }
+};
+
+function createAuthApp() {
+   const app = createApp({
+      initialConfig: {
+         auth: configs.auth
+      }
+   });
+
+   app.emgr.onEvent(
+      App.Events.AppFirstBoot,
+      async () => {
+         await app.createUser(configs.users.normal);
+         await app.createUser(configs.users.admin);
+      },
+      "sync"
+   );
+
+   return app;
+}
+
+function getCookie(r: Response, name: string) {
+   const cookies = r.headers.get("cookie") ?? r.headers.get("set-cookie");
+   if (!cookies) return;
+   const cookie = cookies.split(";").find((c) => c.trim().startsWith(name));
+   if (!cookie) return;
+   return cookie.split("=")[1];
+}
+
+const fns = <Mode extends "cookie" | "token" = "token">(app: App, mode?: Mode) => {
+   function headers(token?: string, additional?: Record<string, string>) {
+      if (mode === "cookie") {
+         return {
+            cookie: `auth=${token};`,
+            ...additional
+         };
+      }
+
+      return {
+         Authorization: `Bearer ${token}`,
+         "Content-Type": "application/json",
+         ...additional
+      };
+   }
+   function body(obj?: Record<string, any>) {
+      if (mode === "cookie") {
+         const formData = new FormData();
+         for (const key in obj) {
+            formData.append(key, obj[key]);
+         }
+         return formData;
+      }
+
+      return JSON.stringify(obj);
+   }
+
+   return {
+      login: async (
+         user: any
+      ): Promise<{ res: Response; data: Mode extends "token" ? AuthResponse : string }> => {
+         const res = (await app.server.request("/api/auth/password/login", {
+            method: "POST",
+            headers: headers(),
+            body: body(user)
+         })) as Response;
+
+         const data = mode === "cookie" ? getCookie(res, "auth") : await res.json();
+
+         return { res, data };
+      },
+      me: async (token?: string): Promise<Pick<AuthResponse, "user">> => {
+         const res = (await app.server.request("/api/auth/me", {
+            method: "GET",
+            headers: headers(token)
+         })) as Response;
+         return await res.json();
+      }
+   };
+};
+
+describe("integration auth", () => {
+   it("should create users on boot", async () => {
+      const app = createAuthApp();
+      await app.build();
+
+      const { data: users } = await app.em.repository("users").findMany();
+      expect(users.length).toBe(2);
+      expect(users[0].email).toBe(configs.users.normal.email);
+      expect(users[1].email).toBe(configs.users.admin.email);
+   });
+
+   it("should log you in with API", async () => {
+      const app = createAuthApp();
+      await app.build();
+      const $fns = fns(app);
+
+      // login api
+      const { data } = await $fns.login(configs.users.normal);
+      const me = await $fns.me(data.token);
+
+      expect(data.user.email).toBe(me.user.email);
+      expect(me.user.email).toBe(configs.users.normal.email);
+
+      // expect no user with no token
+      expect(await $fns.me()).toEqual({ user: null as any });
+
+      // expect no user with invalid token
+      expect(await $fns.me("invalid")).toEqual({ user: null as any });
+      expect(await $fns.me()).toEqual({ user: null as any });
+   });
+
+   it("should log you in with form and cookie", async () => {
+      const app = createAuthApp();
+      await app.build();
+      const $fns = fns(app, "cookie");
+
+      const { res, data: token } = await $fns.login(configs.users.normal);
+      expect(token).toBeDefined();
+      expect(res.status).toBe(302); // because it redirects
+
+      // test cookie jwt interchangability
+      {
+         // expect token to not work as-is for api endpoints
+         expect(await fns(app).me(token)).toEqual({ user: null as any });
+         // hono adds an additional segment to cookies
+         const apified_token = token.split(".").slice(0, -1).join(".");
+         // now it should work
+         // @todo: maybe add a config to don't allow re-use?
+         expect((await fns(app).me(apified_token)).user.email).toBe(configs.users.normal.email);
+      }
+
+      // test cookie with me endpoint
+      {
+         const me = await $fns.me(token);
+         expect(me.user.email).toBe(configs.users.normal.email);
+
+         // check with invalid & empty
+         expect(await $fns.me("invalid")).toEqual({ user: null as any });
+         expect(await $fns.me()).toEqual({ user: null as any });
+      }
+   });
+
+   it("should check for permissions", async () => {
+      const app = createAuthApp();
+      await app.build();
+
+      await withDisabledConsole(async () => {
+         const res = await app.server.request("/api/system/schema");
+         expect(res.status).toBe(403);
+      });
+   });
+});

app/__test__/media/adapters/StorageCloudinaryAdapter.spec.ts

@@ -3,7 +3,7 @@ import { randomString } from "../../../src/core/utils";
 import { StorageCloudinaryAdapter } from "../../../src/media";
 
 import { config } from "dotenv";
-const dotenvOutput = config({ path: `${import.meta.dir}/../../.env` });
+const dotenvOutput = config({ path: `${import.meta.dir}/../../../.env` });
 const {
    CLOUDINARY_CLOUD_NAME,
    CLOUDINARY_API_KEY,

app/__test__/media/adapters/StorageLocalAdapter.spec.ts

@@ -15,7 +15,7 @@ describe("StorageLocalAdapter", () => {
 
    test("puts an object", async () => {
       objects = (await adapter.listObjects()).length;
-      expect(await adapter.putObject(filename, await file.arrayBuffer())).toBeString();
+      expect(await adapter.putObject(filename, file)).toBeString();
    });
 
    test("lists objects", async () => {

app/__test__/media/adapters/StorageS3Adapter.spec.ts

@@ -3,14 +3,14 @@ import { randomString } from "../../../src/core/utils";
 import { StorageS3Adapter } from "../../../src/media";
 
 import { config } from "dotenv";
-const dotenvOutput = config({ path: `${import.meta.dir}/../../.env` });
+const dotenvOutput = config({ path: `${import.meta.dir}/../../../.env` });
 const { R2_ACCESS_KEY, R2_SECRET_ACCESS_KEY, R2_URL, AWS_ACCESS_KEY, AWS_SECRET_KEY, AWS_S3_URL } =
    dotenvOutput.parsed!;
 
 // @todo: mock r2/s3 responses for faster tests
-const ALL_TESTS = process.env.ALL_TESTS;
+const ALL_TESTS = !!process.env.ALL_TESTS;
 
-describe("Storage", async () => {
+describe.skipIf(ALL_TESTS)("StorageS3Adapter", async () => {
    console.log("ALL_TESTS", process.env.ALL_TESTS);
    const versions = [
       [

app/__test__/modules/AppAuth.spec.ts

@@ -1,5 +1,7 @@
-import { afterAll, beforeAll, beforeEach, describe, expect, test } from "bun:test";
+import { afterAll, beforeAll, beforeEach, describe, expect, spyOn, test } from "bun:test";
+import { createApp } from "../../src";
 import { AuthController } from "../../src/auth/api/AuthController";
+import { em, entity, make, text } from "../../src/data";
 import { AppAuth, type ModuleBuildContext } from "../../src/modules";
 import { disableConsoleLog, enableConsoleLog } from "../helper";
 import { makeCtx, moduleTestSuite } from "./module-test-suite";
@@ -76,4 +78,87 @@ describe("AppAuth", () => {
          expect(users[0].email).toBe("some@body.com");
       }
    });
+
+   test("registers auth middleware for bknd routes only", async () => {
+      const app = createApp({
+         initialConfig: {
+            auth: {
+               enabled: true,
+               jwt: {
+                  secret: "123456"
+               }
+            }
+         }
+      });
+
+      await app.build();
+      const spy = spyOn(app.module.auth.authenticator, "requestCookieRefresh");
+
+      // register custom route
+      app.server.get("/test", async (c) => c.text("test"));
+
+      // call a system api and then the custom route
+      await app.server.request("/api/system/ping");
+      await app.server.request("/test");
+
+      expect(spy.mock.calls.length).toBe(1);
+   });
+
+   test("should allow additional user fields", async () => {
+      const app = createApp({
+         initialConfig: {
+            auth: {
+               entity_name: "users",
+               enabled: true
+            },
+            data: em({
+               users: entity("users", {
+                  additional: text()
+               })
+            }).toJSON()
+         }
+      });
+
+      await app.build();
+
+      const e = app.modules.em.entity("users");
+      const fields = e.fields.map((f) => f.name);
+      expect(e.type).toBe("system");
+      expect(fields).toContain("additional");
+      expect(fields).toEqual(["id", "additional", "email", "strategy", "strategy_value", "role"]);
+   });
+
+   test("ensure user field configs is always correct", async () => {
+      const app = createApp({
+         initialConfig: {
+            auth: {
+               enabled: true
+            },
+            data: em({
+               users: entity("users", {
+                  strategy: text({
+                     fillable: true,
+                     hidden: false
+                  }),
+                  strategy_value: text({
+                     fillable: true,
+                     hidden: false
+                  })
+               })
+            }).toJSON()
+         }
+      });
+      await app.build();
+
+      const users = app.em.entity("users");
+      const props = ["hidden", "fillable", "required"];
+
+      for (const [name, _authFieldProto] of Object.entries(AppAuth.usersFields)) {
+         const authField = make(name, _authFieldProto as any);
+         const field = users.field(name)!;
+         for (const prop of props) {
+            expect(field.config[prop]).toBe(authField.config[prop]);
+         }
+      }
+   });
 });

app/__test__/modules/AppMedia.spec.ts

@@ -1,7 +1,55 @@
-import { describe } from "bun:test";
+import { describe, expect, test } from "bun:test";
+import { createApp, registries } from "../../src";
+import { em, entity, text } from "../../src/data";
+import { StorageLocalAdapter } from "../../src/media/storage/adapters/StorageLocalAdapter";
 import { AppMedia } from "../../src/modules";
 import { moduleTestSuite } from "./module-test-suite";
 
 describe("AppMedia", () => {
    moduleTestSuite(AppMedia);
+
+   test("should allow additional fields", async () => {
+      registries.media.register("local", StorageLocalAdapter);
+
+      const app = createApp({
+         initialConfig: {
+            media: {
+               entity_name: "media",
+               enabled: true,
+               adapter: {
+                  type: "local",
+                  config: {
+                     path: "./"
+                  }
+               }
+            },
+            data: em({
+               media: entity("media", {
+                  additional: text()
+               })
+            }).toJSON()
+         }
+      });
+
+      await app.build();
+
+      const e = app.modules.em.entity("media");
+      const fields = e.fields.map((f) => f.name);
+      expect(e.type).toBe("system");
+      expect(fields).toContain("additional");
+      expect(fields).toEqual([
+         "id",
+         "additional",
+         "path",
+         "folder",
+         "mime_type",
+         "size",
+         "scope",
+         "etag",
+         "modified_at",
+         "reference",
+         "entity_id",
+         "metadata"
+      ]);
+   });
 });

app/__test__/modules/Module.spec.ts

@@ -0,0 +1,212 @@
+import { describe, expect, test } from "bun:test";
+import { type TSchema, Type, stripMark } from "../../src/core/utils";
+import { EntityManager, em, entity, index, text } from "../../src/data";
+import { DummyConnection } from "../../src/data/connection/DummyConnection";
+import { Module } from "../../src/modules/Module";
+
+function createModule<Schema extends TSchema>(schema: Schema) {
+   class TestModule extends Module<typeof schema> {
+      getSchema() {
+         return schema;
+      }
+      toJSON() {
+         return this.config;
+      }
+      useForceParse() {
+         return true;
+      }
+   }
+
+   return TestModule;
+}
+
+describe("Module", async () => {
+   describe("basic", () => {
+      test("listener", async () => {
+         let result: any;
+
+         const module = createModule(Type.Object({ a: Type.String() }));
+         const m = new module({ a: "test" });
+
+         await m.schema().set({ a: "test2" });
+         m.setListener(async (c) => {
+            await new Promise((r) => setTimeout(r, 10));
+            result = stripMark(c);
+         });
+         await m.schema().set({ a: "test3" });
+         expect(result).toEqual({ a: "test3" });
+      });
+   });
+
+   describe("db schema", () => {
+      class M extends Module {
+         override getSchema() {
+            return Type.Object({});
+         }
+
+         prt = {
+            ensureEntity: this.ensureEntity.bind(this),
+            ensureIndex: this.ensureIndex.bind(this),
+            ensureSchema: this.ensureSchema.bind(this)
+         };
+
+         get em() {
+            return this.ctx.em;
+         }
+      }
+
+      function make(_em: ReturnType<typeof em>) {
+         const em = new EntityManager(
+            Object.values(_em.entities),
+            new DummyConnection(),
+            _em.relations,
+            _em.indices
+         );
+         return new M({} as any, { em, flags: Module.ctx_flags } as any);
+      }
+      function flat(_em: EntityManager) {
+         return {
+            entities: _em.entities.map((e) => ({
+               name: e.name,
+               fields: e.fields.map((f) => f.name),
+               type: e.type
+            })),
+            indices: _em.indices.map((i) => ({
+               name: i.name,
+               entity: i.entity.name,
+               fields: i.fields.map((f) => f.name),
+               unique: i.unique
+            }))
+         };
+      }
+
+      test("no change", () => {
+         const initial = em({});
+
+         const m = make(initial);
+         expect(m.ctx.flags.sync_required).toBe(false);
+
+         expect(flat(make(initial).em)).toEqual({
+            entities: [],
+            indices: []
+         });
+      });
+
+      test("init", () => {
+         const initial = em({
+            users: entity("u", {
+               name: text()
+            })
+         });
+
+         const m = make(initial);
+         expect(m.ctx.flags.sync_required).toBe(false);
+
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name"],
+                  type: "regular"
+               }
+            ],
+            indices: []
+         });
+      });
+
+      test("ensure entity", () => {
+         const initial = em({
+            users: entity("u", {
+               name: text()
+            })
+         });
+
+         const m = make(initial);
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name"],
+                  type: "regular"
+               }
+            ],
+            indices: []
+         });
+
+         // this should add a new entity
+         m.prt.ensureEntity(
+            entity("p", {
+               title: text()
+            })
+         );
+
+         // this should only add the field "important"
+         m.prt.ensureEntity(
+            entity(
+               "u",
+               {
+                  important: text()
+               },
+               undefined,
+               "system"
+            )
+         );
+
+         expect(m.ctx.flags.sync_required).toBe(true);
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name", "important"],
+                  // ensured type must be present
+                  type: "system"
+               },
+               {
+                  name: "p",
+                  fields: ["id", "title"],
+                  type: "regular"
+               }
+            ],
+            indices: []
+         });
+      });
+
+      test("ensure index", () => {
+         const users = entity("u", {
+            name: text(),
+            title: text()
+         });
+         const initial = em({ users }, ({ index }, { users }) => {
+            index(users).on(["title"]);
+         });
+
+         const m = make(initial);
+         m.prt.ensureIndex(index(users).on(["name"]));
+
+         expect(m.ctx.flags.sync_required).toBe(true);
+         expect(flat(m.em)).toEqual({
+            entities: [
+               {
+                  name: "u",
+                  fields: ["id", "name", "title"],
+                  type: "regular"
+               }
+            ],
+            indices: [
+               {
+                  name: "idx_u_title",
+                  entity: "u",
+                  fields: ["title"],
+                  unique: false
+               },
+               {
+                  name: "idx_u_name",
+                  entity: "u",
+                  fields: ["name"],
+                  unique: false
+               }
+            ]
+         });
+      });
+   });
+});

app/__test__/modules/ModuleManager.spec.ts

@@ -1,9 +1,9 @@
 import { describe, expect, test } from "bun:test";
-import { mark, stripMark } from "../src/core/utils";
-import { entity, text } from "../src/data";
-import { ModuleManager, getDefaultConfig } from "../src/modules/ModuleManager";
-import { CURRENT_VERSION, TABLE_NAME } from "../src/modules/migrations";
-import { getDummyConnection } from "./helper";
+import { stripMark } from "../../src/core/utils";
+import { entity, text } from "../../src/data";
+import { ModuleManager, getDefaultConfig } from "../../src/modules/ModuleManager";
+import { CURRENT_VERSION, TABLE_NAME } from "../../src/modules/migrations";
+import { getDummyConnection } from "../helper";
 
 describe("ModuleManager", async () => {
    test("s1: no config, no build", async () => {

app/__test__/modules/module-test-suite.ts

@@ -2,10 +2,11 @@ import { beforeEach, describe, expect, it } from "bun:test";
 
 import { Hono } from "hono";
 import { Guard } from "../../src/auth";
+import { DebugLogger } from "../../src/core";
 import { EventManager } from "../../src/core/events";
 import { Default, stripMark } from "../../src/core/utils";
 import { EntityManager } from "../../src/data";
-import type { Module, ModuleBuildContext } from "../../src/modules/Module";
+import { Module, type ModuleBuildContext } from "../../src/modules/Module";
 import { getDummyConnection } from "../helper";
 
 export function makeCtx(overrides?: Partial<ModuleBuildContext>): ModuleBuildContext {
@@ -16,6 +17,8 @@ export function makeCtx(overrides?: Partial<ModuleBuildContext>): ModuleBuildCon
       em: new EntityManager([], dummyConnection),
       emgr: new EventManager(),
       guard: new Guard(),
+      flags: Module.ctx_flags,
+      logger: new DebugLogger(false),
       ...overrides
    };
 }

app/__test__/ui/json-form.spec.ts

@@ -0,0 +1,138 @@
+import { describe, expect, test } from "bun:test";
+import { Draft2019 } from "json-schema-library";
+import type { JSONSchema } from "json-schema-to-ts";
+import * as utils from "../../src/ui/components/form/json-schema-form/utils";
+import type { IsTypeType } from "../../src/ui/components/form/json-schema-form/utils";
+
+describe("json form", () => {
+   test("coerse", () => {
+      const examples = [
+         ["test", { type: "string" }, "test"],
+         ["1", { type: "integer" }, 1],
+         ["1", { type: "number" }, 1],
+         ["true", { type: "boolean" }, true],
+         ["false", { type: "boolean" }, false],
+         ["1", { type: "boolean" }, true],
+         ["0", { type: "boolean" }, false],
+         ["on", { type: "boolean" }, true],
+         ["off", { type: "boolean" }, false],
+         ["null", { type: "null" }, null]
+      ] satisfies [string, Exclude<JSONSchema, boolean>, any][];
+
+      for (const [input, schema, output] of examples) {
+         expect(utils.coerce(input, schema)).toBe(output);
+      }
+   });
+
+   test("isType", () => {
+      const examples = [
+         ["string", "string", true],
+         ["integer", "number", false],
+         ["number", "number", true],
+         ["boolean", "boolean", true],
+         ["null", "null", true],
+         ["object", "object", true],
+         ["array", "array", true],
+         ["object", "array", false],
+         [["string", "number"], "number", true],
+         ["number", ["string", "number"], true]
+      ] satisfies [IsTypeType, IsTypeType, boolean][];
+
+      for (const [type, schemaType, output] of examples) {
+         expect(utils.isType(type, schemaType)).toBe(output);
+      }
+   });
+
+   test("getParentPointer", () => {
+      const examples = [
+         ["#/nested/property/0/name", "#/nested/property/0"],
+         ["#/nested/property/0", "#/nested/property"],
+         ["#/nested/property", "#/nested"],
+         ["#/nested", "#"]
+      ];
+
+      for (const [input, output] of examples) {
+         expect(utils.getParentPointer(input)).toBe(output);
+      }
+   });
+
+   test("isRequired", () => {
+      const examples = [
+         [
+            "#/description",
+            { type: "object", properties: { description: { type: "string" } } },
+            false
+         ],
+         [
+            "#/description",
+            {
+               type: "object",
+               required: ["description"],
+               properties: { description: { type: "string" } }
+            },
+            true
+         ],
+         [
+            "#/nested/property",
+            {
+               type: "object",
+               properties: {
+                  nested: {
+                     type: "object",
+                     properties: { property: { type: "string" } }
+                  }
+               }
+            },
+            false
+         ],
+         [
+            "#/nested/property",
+            {
+               type: "object",
+               properties: {
+                  nested: {
+                     type: "object",
+                     required: ["property"],
+                     properties: { property: { type: "string" } }
+                  }
+               }
+            },
+            true
+         ]
+      ] satisfies [string, Exclude<JSONSchema, boolean>, boolean][];
+
+      for (const [pointer, schema, output] of examples) {
+         expect(utils.isRequired(new Draft2019(schema), pointer, schema)).toBe(output);
+      }
+   });
+
+   test("prefixPath", () => {
+      const examples = [
+         ["normal", "", "normal"],
+         ["", "prefix", "prefix"],
+         ["tags", "0", "0.tags"],
+         ["tags", 0, "0.tags"],
+         ["nested.property", "prefix", "prefix.nested.property"],
+         ["nested.property", "", "nested.property"]
+      ] satisfies [string, any, string][];
+
+      for (const [path, prefix, output] of examples) {
+         expect(utils.prefixPath(path, prefix)).toBe(output);
+      }
+   });
+
+   test("suffixPath", () => {
+      const examples = [
+         ["normal", "", "normal"],
+         ["", "suffix", "suffix"],
+         ["tags", "0", "tags.0"],
+         ["tags", 0, "tags.0"],
+         ["nested.property", "suffix", "nested.property.suffix"],
+         ["nested.property", "", "nested.property"]
+      ] satisfies [string, any, string][];
+
+      for (const [path, suffix, output] of examples) {
+         expect(utils.suffixPath(path, suffix)).toBe(output);
+      }
+   });
+});

app/build.ts

@@ -1,8 +1,5 @@
 import { $ } from "bun";
-import * as esbuild from "esbuild";
-import postcss from "esbuild-postcss";
 import * as tsup from "tsup";
-import { guessMimeType } from "./src/media/storage/mime-types";
 
 const args = process.argv.slice(2);
 const watch = args.includes("--watch");
@@ -12,19 +9,21 @@ const sourcemap = args.includes("--sourcemap");
 const clean = args.includes("--clean");
 
 if (clean) {
-   console.log("Cleaning dist");
-   await $`rm -rf dist`;
+   console.log("Cleaning dist (w/o static)");
+   await $`find dist -mindepth 1 ! -path "dist/static/*" ! -path "dist/static" -exec rm -rf {} +`;
 }
 
 let types_running = false;
 function buildTypes() {
-   if (types_running) return;
+   if (types_running || !types) return;
    types_running = true;
 
    Bun.spawn(["bun", "build:types"], {
+      stdout: "inherit",
       onExit: () => {
          console.log("Types built");
          Bun.spawn(["bun", "tsc-alias"], {
+            stdout: "inherit",
             onExit: () => {
                console.log("Types aliased");
                types_running = false;
@@ -36,7 +35,7 @@ function buildTypes() {
 
 let watcher_timeout: any;
 function delayTypes() {
-   if (!watch) return;
+   if (!watch || !types) return;
    if (watcher_timeout) {
       clearTimeout(watcher_timeout);
    }
@@ -47,67 +46,6 @@ if (types && !watch) {
    buildTypes();
 }
 
-/**
- * Build static assets
- * Using esbuild because tsup doesn't include "react"
- */
-const result = await esbuild.build({
-   minify,
-   sourcemap,
-   entryPoints: ["src/ui/main.tsx"],
-   entryNames: "[dir]/[name]-[hash]",
-   outdir: "dist/static",
-   platform: "browser",
-   bundle: true,
-   splitting: true,
-   metafile: true,
-   drop: ["console", "debugger"],
-   inject: ["src/ui/inject.js"],
-   target: "es2022",
-   format: "esm",
-   plugins: [postcss()],
-   loader: {
-      ".svg": "dataurl",
-      ".js": "jsx"
-   },
-   define: {
-      __isDev: "0",
-      "process.env.NODE_ENV": '"production"'
-   },
-   chunkNames: "chunks/[name]-[hash]",
-   logLevel: "error"
-});
-
-// Write manifest
-{
-   const manifest: Record<string, object> = {};
-   const toAsset = (output: string) => {
-      const name = output.split("/").pop()!;
-      return {
-         name,
-         path: output,
-         mime: guessMimeType(name)
-      };
-   };
-
-   const info = Object.entries(result.metafile.outputs)
-      .filter(([, meta]) => {
-         return meta.entryPoint && meta.entryPoint === "src/ui/main.tsx";
-      })
-      .map(([output, meta]) => ({ output, meta }));
-
-   for (const { output, meta } of info) {
-      manifest[meta.entryPoint as string] = toAsset(output);
-      if (meta.cssBundle) {
-         manifest["src/ui/main.css"] = toAsset(meta.cssBundle);
-      }
-   }
-
-   const manifest_file = "dist/static/manifest.json";
-   await Bun.write(manifest_file, JSON.stringify(manifest, null, 2));
-   console.log(`Manifest written to ${manifest_file}`, manifest);
-}
-
 /**
  * Building backend and general API
  */
@@ -120,7 +58,7 @@ await tsup.build({
    external: ["bun:test", "@libsql/client"],
    metafile: true,
    platform: "browser",
-   format: ["esm", "cjs"],
+   format: ["esm"],
    splitting: false,
    treeshake: true,
    loader: {
@@ -138,26 +76,80 @@ await tsup.build({
    minify,
    sourcemap,
    watch,
-   entry: ["src/ui/index.ts", "src/ui/client/index.ts", "src/ui/main.css"],
+   entry: ["src/ui/index.ts", "src/ui/client/index.ts", "src/ui/main.css", "src/ui/styles.css"],
    outDir: "dist/ui",
-   external: ["bun:test", "react", "react-dom", "use-sync-external-store"],
+   external: [
+      "bun:test",
+      "react",
+      "react-dom",
+      "react/jsx-runtime",
+      "react/jsx-dev-runtime",
+      "use-sync-external-store",
+      /codemirror/,
+      "@xyflow/react",
+      "@mantine/core"
+   ],
    metafile: true,
    platform: "browser",
-   format: ["esm", "cjs"],
-   splitting: true,
+   format: ["esm"],
+   splitting: false,
+   bundle: true,
    treeshake: true,
    loader: {
       ".svg": "dataurl"
    },
    esbuildOptions: (options) => {
       options.logLevel = "silent";
-      options.chunkNames = "chunks/[name]-[hash]";
    },
    onSuccess: async () => {
       delayTypes();
    }
 });
 
+/**
+ * Building UI Elements
+ * - tailwind-merge is mocked, no exclude
+ * - ui/client is external, and after built replaced with "bknd/client"
+ */
+await tsup.build({
+   minify,
+   sourcemap,
+   watch,
+   entry: ["src/ui/elements/index.ts"],
+   outDir: "dist/ui/elements",
+   external: [
+      "ui/client",
+      "react",
+      "react-dom",
+      "react/jsx-runtime",
+      "react/jsx-dev-runtime",
+      "use-sync-external-store"
+   ],
+   metafile: true,
+   platform: "browser",
+   format: ["esm"],
+   splitting: false,
+   bundle: true,
+   treeshake: true,
+   loader: {
+      ".svg": "dataurl"
+   },
+   esbuildOptions: (options) => {
+      options.alias = {
+         // not important for elements, mock to reduce bundle
+         "tailwind-merge": "./src/ui/elements/mocks/tailwind-merge.ts"
+      };
+   },
+   onSuccess: async () => {
+      // manually replace ui/client with bknd/client
+      const path = "./dist/ui/elements/index.js";
+      const bundle = await Bun.file(path).text();
+      await Bun.write(path, bundle.replaceAll("ui/client", "bknd/client"));
+
+      delayTypes();
+   }
+});
+
 /**
  * Building adapters
  */
@@ -166,7 +158,7 @@ function baseConfig(adapter: string): tsup.Options {
       minify,
       sourcemap,
       watch,
-      entry: [`src/adapter/${adapter}`],
+      entry: [`src/adapter/${adapter}/index.ts`],
       format: ["esm"],
       platform: "neutral",
       outDir: `dist/adapter/${adapter}`,
@@ -181,44 +173,36 @@ function baseConfig(adapter: string): tsup.Options {
       ],
       metafile: true,
       splitting: false,
-      treeshake: true,
       onSuccess: async () => {
          delayTypes();
       }
    };
 }
 
+// base adapter handles
+await tsup.build({
+   ...baseConfig(""),
+   entry: ["src/adapter/index.ts"],
+   outDir: "dist/adapter"
+});
+
+// specific adatpers
+await tsup.build(baseConfig("remix"));
+await tsup.build(baseConfig("bun"));
+await tsup.build(baseConfig("astro"));
+await tsup.build(baseConfig("cloudflare"));
+
 await tsup.build({
    ...baseConfig("vite"),
    platform: "node"
 });
 
-await tsup.build({
-   ...baseConfig("cloudflare")
-});
-
 await tsup.build({
    ...baseConfig("nextjs"),
-   format: ["esm", "cjs"],
    platform: "node"
 });
 
-await tsup.build({
-   ...baseConfig("remix"),
-   format: ["esm", "cjs"]
-});
-
-await tsup.build({
-   ...baseConfig("bun")
-});
-
 await tsup.build({
    ...baseConfig("node"),
-   platform: "node",
-   format: ["esm", "cjs"]
-});
-
-await tsup.build({
-   ...baseConfig("astro"),
-   format: ["esm", "cjs"]
+   platform: "node"
 });

app/bunfig.toml

@@ -1,2 +1,5 @@
 [install]
-registry = "http://localhost:4873"
+#registry = "http://localhost:4873"
+
+[test]
+coverageSkipTestFiles = true

app/package.json

@@ -3,65 +3,77 @@
   "type": "module",
   "sideEffects": false,
   "bin": "./dist/cli/index.js",
-  "version": "0.4.0",
+  "version": "0.7.1",
+  "description": "Lightweight Firebase/Supabase alternative built to run anywhere — incl. Next.js, Remix, Astro, Cloudflare, Bun, Node, AWS Lambda & more.",
+  "homepage": "https://bknd.io",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/bknd-io/bknd.git"
+  },
+  "bugs": {
+    "url": "https://github.com/bknd-io/bknd/issues"
+  },
   "scripts": {
-    "build:all": "NODE_ENV=production bun run build.ts --minify --types --clean && bun run build:cli",
     "dev": "vite",
     "test": "ALL_TESTS=1 bun test --bail",
+    "test:coverage": "ALL_TESTS=1 bun test --bail --coverage",
     "build": "NODE_ENV=production bun run build.ts --minify --types",
+    "build:all": "rm -rf dist && bun run build:static && NODE_ENV=production bun run build.ts --minify --types --clean && bun run build:cli",
+    "build:cli": "bun build src/cli/index.ts --target node --outdir dist/cli --minify",
+    "build:static": "vite build",
     "watch": "bun run build.ts --types --watch",
     "types": "bun tsc --noEmit",
     "clean:types": "find ./dist -name '*.d.ts' -delete && rm -f ./dist/tsconfig.tsbuildinfo",
     "build:types": "tsc --emitDeclarationOnly && tsc-alias",
-    "build:css": "bun tailwindcss -i src/ui/main.css -o ./dist/static/styles.css",
-    "watch:css": "bun tailwindcss --watch -i src/ui/main.css -o ./dist/styles.css",
     "updater": "bun x npm-check-updates -ui",
-    "build:cli": "bun build src/cli/index.ts --target node --outdir dist/cli --minify",
     "cli": "LOCAL=1 bun src/cli/index.ts",
-    "prepublishOnly": "bun run build:all"
+    "prepublishOnly": "bun run types && bun run test && bun run build:all && cp ../README.md ./",
+    "postpublish": "rm -f README.md"
   },
   "license": "FSL-1.1-MIT",
   "dependencies": {
     "@cfworker/json-schema": "^2.0.1",
+    "@codemirror/lang-html": "^6.4.9",
+    "@codemirror/lang-json": "^6.0.1",
+    "@codemirror/lang-liquid": "^6.2.1",
+    "@hello-pangea/dnd": "^17.0.0",
     "@libsql/client": "^0.14.0",
+    "@mantine/core": "^7.13.4",
     "@sinclair/typebox": "^0.32.34",
     "@tanstack/react-form": "0.19.2",
+    "@uiw/react-codemirror": "^4.23.6",
+    "@xyflow/react": "^12.3.2",
     "aws4fetch": "^1.0.18",
     "dayjs": "^1.11.13",
     "fast-xml-parser": "^4.4.0",
     "hono": "^4.6.12",
+    "json-schema-form-react": "^0.0.2",
+    "json-schema-library": "^10.0.0-rc7",
     "kysely": "^0.27.4",
     "liquidjs": "^10.15.0",
     "lodash-es": "^4.17.21",
     "oauth4webapi": "^2.11.1",
+    "object-path-immutable": "^4.1.2",
+    "radix-ui": "^1.1.2",
+    "json-schema-to-ts": "^3.1.1",
     "swr": "^2.2.5"
   },
   "devDependencies": {
     "@aws-sdk/client-s3": "^3.613.0",
-    "@codemirror/lang-html": "^6.4.9",
-    "@codemirror/lang-json": "^6.0.1",
-    "@codemirror/lang-liquid": "^6.2.1",
     "@dagrejs/dagre": "^1.1.4",
-    "@hello-pangea/dnd": "^17.0.0",
     "@hono/typebox-validator": "^0.2.6",
     "@hono/vite-dev-server": "^0.17.0",
     "@hono/zod-validator": "^0.4.1",
     "@hookform/resolvers": "^3.9.1",
     "@libsql/kysely-libsql": "^0.4.1",
-    "@mantine/core": "^7.13.4",
-    "@mantine/hooks": "^7.13.4",
-    "@mantine/modals": "^7.13.4",
-    "@mantine/notifications": "^7.13.5",
-    "@radix-ui/react-scroll-area": "^1.2.0",
     "@rjsf/core": "^5.22.2",
     "@tabler/icons-react": "3.18.0",
     "@types/node": "^22.10.0",
     "@types/react": "^18.3.12",
     "@types/react-dom": "^18.3.1",
-    "@uiw/react-codemirror": "^4.23.6",
     "@vitejs/plugin-react": "^4.3.3",
-    "@xyflow/react": "^12.3.2",
     "autoprefixer": "^10.4.20",
+    "clsx": "^2.1.1",
     "esbuild-postcss": "^0.0.4",
     "jotai": "^2.10.1",
     "open": "^10.1.0",
@@ -72,6 +84,7 @@
     "react-hook-form": "^7.53.1",
     "react-icons": "5.2.1",
     "react-json-view-lite": "^2.0.1",
+    "sql-formatter": "^15.4.9",
     "tailwind-merge": "^2.5.4",
     "tailwindcss": "^3.4.14",
     "tailwindcss-animate": "^1.0.7",
@@ -103,6 +116,11 @@
       "import": "./dist/ui/index.js",
       "require": "./dist/ui/index.cjs"
     },
+    "./elements": {
+      "types": "./dist/types/ui/elements/index.d.ts",
+      "import": "./dist/ui/elements/index.js",
+      "require": "./dist/ui/elements/index.cjs"
+    },
     "./client": {
       "types": "./dist/types/ui/client/index.d.ts",
       "import": "./dist/ui/client/index.js",
@@ -133,6 +151,10 @@
       "import": "./dist/adapter/cloudflare/index.js",
       "require": "./dist/adapter/cloudflare/index.cjs"
     },
+    "./adapter": {
+      "types": "./dist/types/adapter/index.d.ts",
+      "import": "./dist/adapter/index.js"
+    },
     "./adapter/vite": {
       "types": "./dist/types/adapter/vite/index.d.ts",
       "import": "./dist/adapter/vite/index.js",
@@ -163,8 +185,9 @@
       "import": "./dist/adapter/astro/index.js",
       "require": "./dist/adapter/astro/index.cjs"
     },
-    "./dist/styles.css": "./dist/ui/main.css",
-    "./dist/manifest.json": "./dist/static/manifest.json"
+    "./dist/main.css": "./dist/ui/main.css",
+    "./dist/styles.css": "./dist/ui/styles.css",
+    "./dist/manifest.json": "./dist/static/.vite/manifest.json"
   },
   "publishConfig": {
     "access": "public"
@@ -177,5 +200,21 @@
     "!dist/**/*.map",
     "!dist/metafile*",
     "!dist/**/metafile*"
+  ],
+  "keywords": [
+    "backend",
+    "database",
+    "authentication",
+    "media",
+    "workflows",
+    "api",
+    "jwt",
+    "serverless",
+    "cloudflare",
+    "nextjs",
+    "remix",
+    "astro",
+    "bun",
+    "node"
   ]
 }

app/src/Api.ts

@@ -17,13 +17,21 @@ declare global {
 }
 
 export type ApiOptions = {
-   host: string;
-   user?: TApiUser;
-   token?: string;
+   host?: string;
    headers?: Headers;
    key?: string;
    localStorage?: boolean;
-};
+   fetcher?: typeof fetch;
+   verified?: boolean;
+} & (
+   | {
+        token?: string;
+        user?: TApiUser;
+     }
+   | {
+        request: Request;
+     }
+);
 
 export type AuthState = {
    token?: string;
@@ -42,14 +50,26 @@ export class Api {
    public auth!: AuthApi;
    public media!: MediaApi;
 
-   constructor(private readonly options: ApiOptions) {
-      if (options.user) {
-         this.user = options.user;
-         this.token_transport = "none";
-         this.verified = true;
-      } else if (options.token) {
+   constructor(private options: ApiOptions = {}) {
+      // only mark verified if forced
+      this.verified = options.verified === true;
+
+      // prefer request if given
+      if ("request" in options) {
+         this.options.host = options.host ?? new URL(options.request.url).origin;
+         this.options.headers = options.headers ?? options.request.headers;
+         this.extractToken();
+
+         // then check for a token
+      } else if ("token" in options) {
          this.token_transport = "header";
          this.updateToken(options.token);
+
+         // then check for an user object
+      } else if ("user" in options) {
+         this.token_transport = "none";
+         this.user = options.user;
+         this.verified = options.verified !== false;
       } else {
          this.extractToken();
       }
@@ -58,7 +78,7 @@ export class Api {
    }
 
    get baseUrl() {
-      return this.options.host;
+      return this.options.host ?? "http://localhost";
    }
 
    get tokenKey() {
@@ -66,13 +86,15 @@ export class Api {
    }
 
    private extractToken() {
+      // if token has to be extracted, it's never verified
+      this.verified = false;
+
       if (this.options.headers) {
          // try cookies
          const cookieToken = getCookieValue(this.options.headers.get("cookie"), "auth");
          if (cookieToken) {
-            this.updateToken(cookieToken);
             this.token_transport = "cookie";
-            this.verified = true;
+            this.updateToken(cookieToken);
             return;
          }
 
@@ -96,6 +118,8 @@ export class Api {
 
    updateToken(token?: string, rebuild?: boolean) {
       this.token = token;
+      this.verified = false;
+
       if (token) {
          this.user = omit(decode(token).payload as any, ["iat", "iss", "exp"]) as any;
       } else {
@@ -115,11 +139,15 @@ export class Api {
       if (rebuild) this.buildApis();
    }
 
-   markAuthVerified(verfied: boolean) {
+   private markAuthVerified(verfied: boolean) {
       this.verified = verfied;
       return this;
    }
 
+   isAuthVerified(): boolean {
+      return this.verified;
+   }
+
    getAuthState(): AuthState {
       return {
          token: this.token,
@@ -128,21 +156,30 @@ export class Api {
       };
    }
 
-   async getVerifiedAuthState(force?: boolean): Promise<AuthState> {
-      if (force === true || !this.verified) {
-         await this.verifyAuth();
-      }
+   isAuthenticated(): boolean {
+      const { token, user } = this.getAuthState();
+      return !!token && !!user;
+   }
 
+   async getVerifiedAuthState(): Promise<AuthState> {
+      await this.verifyAuth();
       return this.getAuthState();
    }
 
    async verifyAuth() {
+      if (!this.token) {
+         this.markAuthVerified(false);
+         return;
+      }
+
       try {
-         const res = await this.auth.me();
-         if (!res.ok || !res.body.user) {
+         const { ok, data } = await this.auth.me();
+         const user = data?.user;
+         if (!ok || !user) {
             throw new Error();
          }
 
+         this.user = user;
          this.markAuthVerified(true);
       } catch (e) {
          this.markAuthVerified(false);
@@ -154,21 +191,29 @@ export class Api {
       return this.user || null;
    }
 
-   private buildApis() {
-      const baseParams = {
-         host: this.options.host,
+   getParams() {
+      return Object.freeze({
+         host: this.baseUrl,
          token: this.token,
          headers: this.options.headers,
          token_transport: this.token_transport
-      };
-
-      this.system = new SystemApi(baseParams);
-      this.data = new DataApi(baseParams);
-      this.auth = new AuthApi({
-         ...baseParams,
-         onTokenUpdate: (token) => this.updateToken(token, true)
       });
-      this.media = new MediaApi(baseParams);
+   }
+
+   private buildApis() {
+      const baseParams = this.getParams();
+      const fetcher = this.options.fetcher;
+
+      this.system = new SystemApi(baseParams, fetcher);
+      this.data = new DataApi(baseParams, fetcher);
+      this.auth = new AuthApi(
+         {
+            ...baseParams,
+            onTokenUpdate: (token) => this.updateToken(token, true)
+         },
+         fetcher
+      );
+      this.media = new MediaApi(baseParams, fetcher);
    }
 }
 

app/src/App.ts

@@ -1,5 +1,7 @@
+import type { CreateUserPayload } from "auth/AppAuth";
 import { Event } from "core/events";
 import { Connection, type LibSqlCredentials, LibsqlConnection } from "data";
+import type { Hono } from "hono";
 import {
    type InitialModuleConfigs,
    ModuleManager,
@@ -68,6 +70,12 @@ export class App {
          onFirstBoot: async () => {
             console.log("[APP] first boot");
             this.trigger_first_boot = true;
+         },
+         onServerInit: async (server) => {
+            server.use(async (c, next) => {
+               c.set("app", this);
+               await next();
+            });
          }
       });
       this.modules.ctx().emgr.registerEvents(AppEvents);
@@ -87,20 +95,20 @@ export class App {
          //console.log("syncing", syncResult);
       }
 
+      const { guard, server } = this.modules.ctx();
+
       // load system controller
-      this.modules.ctx().guard.registerPermissions(Object.values(SystemPermissions));
-      this.modules.server.route("/api/system", new SystemController(this).getController());
+      guard.registerPermissions(Object.values(SystemPermissions));
+      server.route("/api/system", new SystemController(this).getController());
 
       // load plugins
       if (this.plugins.length > 0) {
          await Promise.all(this.plugins.map((plugin) => plugin(this)));
       }
 
-      //console.log("emitting built", options);
       await this.emgr.emit(new AppBuiltEvent({ app: this }));
 
-      // not found on any not registered api route
-      this.modules.server.all("/api/*", async (c) => c.notFound());
+      server.all("/api/*", async (c) => c.notFound());
 
       if (options?.save) {
          await this.modules.save();
@@ -121,7 +129,11 @@ export class App {
       return this.modules.server;
    }
 
-   get fetch(): any {
+   get em() {
+      return this.modules.ctx().em;
+   }
+
+   get fetch(): Hono["fetch"] {
       return this.server.fetch;
    }
 
@@ -144,10 +156,14 @@ export class App {
       return this.modules.version();
    }
 
+   isBuilt(): boolean {
+      return this.modules.isBuilt();
+   }
+
    registerAdminController(config?: AdminControllerOptions) {
       // register admin
       this.adminController = new AdminController(this, config);
-      this.modules.server.route("/", this.adminController.getController());
+      this.modules.server.route(config?.basepath ?? "/", this.adminController.getController());
       return this;
    }
 
@@ -158,6 +174,10 @@ export class App {
    static create(config: CreateAppConfig) {
       return createApp(config);
    }
+
+   async createUser(p: CreateUserPayload) {
+      return this.module.auth.createUser(p);
+   }
 }
 
 export function createApp(config: CreateAppConfig = {}) {

app/src/adapter/astro/astro.adapter.ts

@@ -1,7 +1,8 @@
-import { type FrameworkBkndConfig, createFrameworkApp } from "adapter";
-import { Api, type ApiOptions, type App } from "bknd";
+import type { App } from "bknd";
+import { type FrameworkBkndConfig, createFrameworkApp } from "bknd/adapter";
+import { Api, type ApiOptions } from "bknd/client";
 
-export type AstroBkndConfig = FrameworkBkndConfig;
+export type AstroBkndConfig<Args = TAstro> = FrameworkBkndConfig<Args>;
 
 type TAstro = {
    request: Request;
@@ -13,18 +14,20 @@ export type Options = {
       host?: string;
    };
 
-export function getApi(Astro: TAstro, options: Options = { mode: "static" }) {
-   return new Api({
+export async function getApi(Astro: TAstro, options: Options = { mode: "static" }) {
+   const api = new Api({
       host: new URL(Astro.request.url).origin,
       headers: options.mode === "dynamic" ? Astro.request.headers : undefined
    });
+   await api.verifyAuth();
+   return api;
 }
 
 let app: App;
-export function serve(config: AstroBkndConfig = {}) {
-   return async (args: TAstro) => {
+export function serve<Context extends TAstro = TAstro>(config: AstroBkndConfig<Context> = {}) {
+   return async (args: Context) => {
       if (!app) {
-         app = await createFrameworkApp(config);
+         app = await createFrameworkApp(config, args);
       }
       return app.fetch(args.request);
    };

app/src/adapter/bun/bun.adapter.ts

@@ -2,28 +2,23 @@
 
 import path from "node:path";
 import type { App } from "bknd";
+import { type RuntimeBkndConfig, createRuntimeApp } from "bknd/adapter";
+import { registerLocalMediaAdapter } from "bknd/adapter/node";
+import { config } from "bknd/core";
 import type { ServeOptions } from "bun";
-import { config } from "core";
 import { serveStatic } from "hono/bun";
-import { type RuntimeBkndConfig, createRuntimeApp } from "../index";
 
 let app: App;
 
 export type BunBkndConfig = RuntimeBkndConfig & Omit<ServeOptions, "fetch">;
 
-export async function createApp({
-   distPath,
-   onBuilt,
-   buildConfig,
-   beforeBuild,
-   ...config
-}: RuntimeBkndConfig = {}) {
+export async function createApp({ distPath, ...config }: RuntimeBkndConfig = {}) {
    const root = path.resolve(distPath ?? "./node_modules/bknd/dist", "static");
 
    if (!app) {
+      registerLocalMediaAdapter();
       app = await createRuntimeApp({
          ...config,
-         registerLocalMedia: true,
          serveStatic: serveStatic({ root })
       });
    }

app/src/adapter/cloudflare/cloudflare-workers.adapter.ts

@@ -1,18 +1,17 @@
-import type { CreateAppConfig } from "bknd";
+import type { FrameworkBkndConfig } from "bknd/adapter";
 import { Hono } from "hono";
 import { serveStatic } from "hono/cloudflare-workers";
-import type { FrameworkBkndConfig } from "../index";
 import { getCached } from "./modes/cached";
 import { getDurable } from "./modes/durable";
 import { getFresh, getWarm } from "./modes/fresh";
 
-export type CloudflareBkndConfig<Env = any> = Omit<FrameworkBkndConfig, "app"> & {
-   app: CreateAppConfig | ((env: Env) => CreateAppConfig);
+export type CloudflareBkndConfig<Env = any> = FrameworkBkndConfig<Context<Env>> & {
    mode?: "warm" | "fresh" | "cache" | "durable";
-   bindings?: (env: Env) => {
+   bindings?: (args: Context<Env>) => {
       kv?: KVNamespace;
       dobj?: DurableObjectNamespace;
    };
+   static?: "kv" | "assets";
    key?: string;
    keepAliveSeconds?: number;
    forceHttps?: boolean;
@@ -21,28 +20,33 @@ export type CloudflareBkndConfig<Env = any> = Omit<FrameworkBkndConfig, "app"> &
    html?: string;
 };
 
-export type Context = {
+export type Context<Env = any> = {
    request: Request;
-   env: any;
+   env: Env;
    ctx: ExecutionContext;
 };
 
-export function serve(config: CloudflareBkndConfig) {
+export function serve<Env = any>(config: CloudflareBkndConfig<Env>) {
    return {
-      async fetch(request: Request, env: any, ctx: ExecutionContext) {
+      async fetch(request: Request, env: Env, ctx: ExecutionContext) {
          const url = new URL(request.url);
-         const manifest = config.manifest;
 
-         if (manifest) {
+         if (config.manifest && config.static === "assets") {
+            console.warn("manifest is not useful with static 'assets'");
+         } else if (!config.manifest && config.static === "kv") {
+            throw new Error("manifest is required with static 'kv'");
+         }
+
+         if (config.manifest && config.static !== "assets") {
             const pathname = url.pathname.slice(1);
-            const assetManifest = JSON.parse(manifest);
+            const assetManifest = JSON.parse(config.manifest);
             if (pathname && pathname in assetManifest) {
                const hono = new Hono();
 
                hono.all("*", async (c, next) => {
                   const res = await serveStatic({
                      path: `./${pathname}`,
-                     manifest
+                     manifest: config.manifest!
                   })(c as any, next);
                   if (res instanceof Response) {
                      const ttl = 60 * 60 * 24 * 365;

app/src/adapter/cloudflare/modes/cached.ts

@@ -1,8 +1,8 @@
-import { createRuntimeApp } from "adapter";
 import { App } from "bknd";
+import { createRuntimeApp } from "bknd/adapter";
 import type { CloudflareBkndConfig, Context } from "../index";
 
-export async function getCached(config: CloudflareBkndConfig, { env, ctx }: Context) {
+export async function getCached(config: CloudflareBkndConfig, { env, ctx, ...args }: Context) {
    const { kv } = config.bindings?.(env)!;
    if (!kv) throw new Error("kv namespace is not defined in cloudflare.bindings");
    const key = config.key ?? "app";
@@ -37,7 +37,7 @@ export async function getCached(config: CloudflareBkndConfig, { env, ctx }: Cont
          },
          adminOptions: { html: config.html }
       },
-      env
+      { env, ctx, ...args }
    );
 
    if (!cachedConfig) {

app/src/adapter/cloudflare/modes/durable.ts

@@ -1,7 +1,7 @@
 import { DurableObject } from "cloudflare:workers";
-import { createRuntimeApp } from "adapter";
-import type { CloudflareBkndConfig, Context } from "adapter/cloudflare";
 import type { App, CreateAppConfig } from "bknd";
+import { createRuntimeApp, makeConfig } from "bknd/adapter";
+import type { CloudflareBkndConfig, Context } from "../index";
 
 export async function getDurable(config: CloudflareBkndConfig, ctx: Context) {
    const { dobj } = config.bindings?.(ctx.env)!;
@@ -17,7 +17,7 @@ export async function getDurable(config: CloudflareBkndConfig, ctx: Context) {
    const id = dobj.idFromName(key);
    const stub = dobj.get(id) as unknown as DurableBkndApp;
 
-   const create_config = typeof config.app === "function" ? config.app(ctx.env) : config.app;
+   const create_config = makeConfig(config, ctx);
 
    const res = await stub.fire(ctx.request, {
       config: create_config,

app/src/adapter/cloudflare/modes/fresh.ts

@@ -1,14 +1,14 @@
-import { createRuntimeApp } from "adapter";
 import type { App } from "bknd";
+import { createRuntimeApp } from "bknd/adapter";
 import type { CloudflareBkndConfig, Context } from "../index";
 
-export async function makeApp(config: CloudflareBkndConfig, { env }: Context) {
+export async function makeApp(config: CloudflareBkndConfig, ctx: Context) {
    return await createRuntimeApp(
       {
          ...config,
          adminOptions: config.html ? { html: config.html } : undefined
       },
-      env
+      ctx
    );
 }
 

app/src/adapter/index.ts

@@ -1,58 +1,29 @@
-import type { IncomingMessage } from "node:http";
-import { App, type CreateAppConfig, registries } from "bknd";
+import { App, type CreateAppConfig } from "bknd";
+import { config as $config } from "bknd/core";
 import type { MiddlewareHandler } from "hono";
-import { StorageLocalAdapter } from "media/storage/adapters/StorageLocalAdapter";
 import type { AdminControllerOptions } from "modules/server/AdminController";
 
-export type BkndConfig<Env = any> = CreateAppConfig & {
-   app?: CreateAppConfig | ((env: Env) => CreateAppConfig);
+export type BkndConfig<Args = any> = CreateAppConfig & {
+   app?: CreateAppConfig | ((args: Args) => CreateAppConfig);
    onBuilt?: (app: App) => Promise<void>;
    beforeBuild?: (app: App) => Promise<void>;
    buildConfig?: Parameters<App["build"]>[0];
 };
 
-export type FrameworkBkndConfig<Env = any> = BkndConfig<Env>;
+export type FrameworkBkndConfig<Args = any> = BkndConfig<Args>;
 
-export type RuntimeBkndConfig<Env = any> = BkndConfig<Env> & {
+export type RuntimeBkndConfig<Args = any> = BkndConfig<Args> & {
    distPath?: string;
 };
 
-export function nodeRequestToRequest(req: IncomingMessage): Request {
-   let protocol = "http";
-   try {
-      protocol = req.headers["x-forwarded-proto"] as string;
-   } catch (e) {}
-   const host = req.headers.host;
-   const url = `${protocol}://${host}${req.url}`;
-   const headers = new Headers();
-
-   for (const [key, value] of Object.entries(req.headers)) {
-      if (Array.isArray(value)) {
-         headers.append(key, value.join(", "));
-      } else if (value) {
-         headers.append(key, value);
-      }
-   }
-
-   const method = req.method || "GET";
-   return new Request(url, {
-      method,
-      headers
-   });
-}
-
-export function registerLocalMediaAdapter() {
-   registries.media.register("local", StorageLocalAdapter);
-}
-
-export function makeConfig<Env = any>(config: BkndConfig<Env>, env?: Env): CreateAppConfig {
+export function makeConfig<Args = any>(config: BkndConfig<Args>, args?: Args): CreateAppConfig {
    let additionalConfig: CreateAppConfig = {};
    if ("app" in config && config.app) {
       if (typeof config.app === "function") {
-         if (!env) {
-            throw new Error("env is required when config.app is a function");
+         if (!args) {
+            throw new Error("args is required when config.app is a function");
          }
-         additionalConfig = config.app(env);
+         additionalConfig = config.app(args);
       } else {
          additionalConfig = config.app;
       }
@@ -61,11 +32,11 @@ export function makeConfig<Env = any>(config: BkndConfig<Env>, env?: Env): Creat
    return { ...config, ...additionalConfig };
 }
 
-export async function createFrameworkApp<Env = any>(
+export async function createFrameworkApp<Args = any>(
    config: FrameworkBkndConfig,
-   env?: Env
+   args?: Args
 ): Promise<App> {
-   const app = App.create(makeConfig(config, env));
+   const app = App.create(makeConfig(config, args));
 
    if (config.onBuilt) {
       app.emgr.onEvent(
@@ -86,32 +57,24 @@ export async function createFrameworkApp<Env = any>(
 export async function createRuntimeApp<Env = any>(
    {
       serveStatic,
-      registerLocalMedia,
       adminOptions,
       ...config
    }: RuntimeBkndConfig & {
       serveStatic?: MiddlewareHandler | [string, MiddlewareHandler];
-      registerLocalMedia?: boolean;
       adminOptions?: AdminControllerOptions | false;
    },
    env?: Env
 ): Promise<App> {
-   if (registerLocalMedia) {
-      registerLocalMediaAdapter();
-   }
-
    const app = App.create(makeConfig(config, env));
 
    app.emgr.onEvent(
       App.Events.AppBuiltEvent,
       async () => {
          if (serveStatic) {
-            if (Array.isArray(serveStatic)) {
-               const [path, handler] = serveStatic;
-               app.modules.server.get(path, handler);
-            } else {
-               app.modules.server.get("/*", serveStatic);
-            }
+            const [path, handler] = Array.isArray(serveStatic)
+               ? serveStatic
+               : [$config.server.assets_path + "*", serveStatic];
+            app.modules.server.get(path, handler);
          }
 
          await config.onBuilt?.(app);

app/src/adapter/nextjs/nextjs.adapter.ts

@@ -1,8 +1,12 @@
 import type { IncomingMessage, ServerResponse } from "node:http";
-import { Api, type App } from "bknd";
-import { type FrameworkBkndConfig, createFrameworkApp, nodeRequestToRequest } from "../index";
+import { nodeRequestToRequest } from "adapter/utils";
+import type { App } from "bknd";
+import { type FrameworkBkndConfig, createFrameworkApp } from "bknd/adapter";
+import { Api } from "bknd/client";
 
-export type NextjsBkndConfig = FrameworkBkndConfig;
+export type NextjsBkndConfig = FrameworkBkndConfig & {
+   cleanSearch?: string[];
+};
 
 type GetServerSidePropsContext = {
    req: IncomingMessage;
@@ -27,15 +31,20 @@ export function createApi({ req }: GetServerSidePropsContext) {
 }
 
 export function withApi<T>(handler: (ctx: GetServerSidePropsContext & { api: Api }) => T) {
-   return (ctx: GetServerSidePropsContext & { api: Api }) => {
-      return handler({ ...ctx, api: createApi(ctx) });
+   return async (ctx: GetServerSidePropsContext & { api: Api }) => {
+      const api = createApi(ctx);
+      await api.verifyAuth();
+      return handler({ ...ctx, api });
    };
 }
 
-function getCleanRequest(req: Request) {
-   // clean search params from "route" attribute
+function getCleanRequest(
+   req: Request,
+   { cleanSearch = ["route"] }: Pick<NextjsBkndConfig, "cleanSearch">
+) {
    const url = new URL(req.url);
-   url.searchParams.delete("route");
+   cleanSearch?.forEach((k) => url.searchParams.delete(k));
+
    return new Request(url.toString(), {
       method: req.method,
       headers: req.headers,
@@ -44,12 +53,12 @@ function getCleanRequest(req: Request) {
 }
 
 let app: App;
-export function serve(config: NextjsBkndConfig = {}) {
+export function serve({ cleanSearch, ...config }: NextjsBkndConfig = {}) {
    return async (req: Request) => {
       if (!app) {
          app = await createFrameworkApp(config);
       }
-      const request = getCleanRequest(req);
+      const request = getCleanRequest(req, { cleanSearch });
       return app.fetch(request, process.env);
    };
 }

app/src/adapter/node/index.ts

@@ -1,6 +1,12 @@
-export * from "./node.adapter";
-export {
-   StorageLocalAdapter,
-   type LocalAdapterConfig
+import { registries } from "bknd";
+import {
+   type LocalAdapterConfig,
+   StorageLocalAdapter
 } from "../../media/storage/adapters/StorageLocalAdapter";
-export { registerLocalMediaAdapter } from "../index";
+
+export * from "./node.adapter";
+export { StorageLocalAdapter, type LocalAdapterConfig };
+
+export function registerLocalMediaAdapter() {
+   registries.media.register("local", StorageLocalAdapter);
+}

app/src/adapter/node/node.adapter.ts

@@ -1,9 +1,10 @@
 import path from "node:path";
 import { serve as honoServe } from "@hono/node-server";
 import { serveStatic } from "@hono/node-server/serve-static";
+import { registerLocalMediaAdapter } from "adapter/node/index";
 import type { App } from "bknd";
-import { config as $config } from "core";
-import { type RuntimeBkndConfig, createRuntimeApp } from "../index";
+import { type RuntimeBkndConfig, createRuntimeApp } from "bknd/adapter";
+import { config as $config } from "bknd/core";
 
 export type NodeBkndConfig = RuntimeBkndConfig & {
    port?: number;
@@ -19,9 +20,6 @@ export function serve({
    port = $config.server.default_port,
    hostname,
    listener,
-   onBuilt,
-   buildConfig = {},
-   beforeBuild,
    ...config
 }: NodeBkndConfig = {}) {
    const root = path.relative(
@@ -40,9 +38,9 @@ export function serve({
          hostname,
          fetch: async (req: Request) => {
             if (!app) {
+               registerLocalMediaAdapter();
                app = await createRuntimeApp({
                   ...config,
-                  registerLocalMedia: true,
                   serveStatic: serveStatic({ root })
                });
             }

app/src/adapter/remix/AdminPage.tsx

@@ -1,9 +1,11 @@
+import { useAuth } from "bknd/client";
 import type { BkndAdminProps } from "bknd/ui";
 import { Suspense, lazy, useEffect, useState } from "react";
 
 export function adminPage(props?: BkndAdminProps) {
    const Admin = lazy(() => import("bknd/ui").then((mod) => ({ default: mod.Admin })));
    return () => {
+      const auth = useAuth();
       const [loaded, setLoaded] = useState(false);
       useEffect(() => {
          if (typeof window === "undefined") return;
@@ -13,7 +15,7 @@ export function adminPage(props?: BkndAdminProps) {
 
       return (
          <Suspense>
-            <Admin {...props} />
+            <Admin withProvider={{ user: auth.user }} {...props} />
          </Suspense>
       );
    };

app/src/adapter/remix/remix.adapter.ts

@@ -1,14 +1,37 @@
-import { type FrameworkBkndConfig, createFrameworkApp } from "adapter";
 import type { App } from "bknd";
+import { type FrameworkBkndConfig, createFrameworkApp } from "bknd/adapter";
+import { Api } from "bknd/client";
 
-export type RemixBkndConfig = FrameworkBkndConfig;
+export type RemixBkndConfig<Args = RemixContext> = FrameworkBkndConfig<Args>;
+
+type RemixContext = {
+   request: Request;
+};
 
 let app: App;
-export function serve(config: RemixBkndConfig = {}) {
-   return async (args: { request: Request }) => {
+export function serve<Args extends RemixContext = RemixContext>(
+   config: RemixBkndConfig<Args> = {}
+) {
+   return async (args: Args) => {
       if (!app) {
-         app = await createFrameworkApp(config);
+         app = await createFrameworkApp(config, args);
       }
       return app.fetch(args.request);
    };
 }
+
+export function withApi<Args extends { request: Request; context: { api: Api } }, R>(
+   handler: (args: Args, api: Api) => Promise<R>
+) {
+   return async (args: Args) => {
+      if (!args.context.api) {
+         args.context.api = new Api({
+            host: new URL(args.request.url).origin,
+            headers: args.request.headers
+         });
+         await args.context.api.verifyAuth();
+      }
+
+      return handler(args, args.context.api);
+   };
+}

app/src/adapter/utils.ts

@@ -0,0 +1,25 @@
+import type { IncomingMessage } from "node:http";
+
+export function nodeRequestToRequest(req: IncomingMessage): Request {
+   let protocol = "http";
+   try {
+      protocol = req.headers["x-forwarded-proto"] as string;
+   } catch (e) {}
+   const host = req.headers.host;
+   const url = `${protocol}://${host}${req.url}`;
+   const headers = new Headers();
+
+   for (const [key, value] of Object.entries(req.headers)) {
+      if (Array.isArray(value)) {
+         headers.append(key, value.join(", "));
+      } else if (value) {
+         headers.append(key, value);
+      }
+   }
+
+   const method = req.method || "GET";
+   return new Request(url, {
+      method,
+      headers
+   });
+}

app/src/adapter/vite/dev-server-config.ts

@@ -0,0 +1,14 @@
+export const devServerConfig = {
+   entry: "./server.ts",
+   exclude: [
+      /.*\.tsx?($|\?)/,
+      /^(?!.*\/__admin).*\.(s?css|less)($|\?)/,
+      // exclude except /api
+      /^(?!.*\/api).*\.(ico|mp4|jpg|jpeg|svg|png|vtt|mp3|js)($|\?)/,
+      /^\/@.+$/,
+      /\/components.*?\.json.*/, // @todo: improve
+      /^\/(public|assets|static)\/.+/,
+      /^\/node_modules\/.*/
+   ] as any,
+   injectClientScript: false
+} as const;

app/src/adapter/vite/vite.adapter.ts

@@ -1,10 +1,14 @@
 import { serveStatic } from "@hono/node-server/serve-static";
-import { type RuntimeBkndConfig, createRuntimeApp } from "adapter";
+import { type DevServerOptions, default as honoViteDevServer } from "@hono/vite-dev-server";
 import type { App } from "bknd";
+import { type RuntimeBkndConfig, createRuntimeApp } from "bknd/adapter";
+import { registerLocalMediaAdapter } from "bknd/adapter/node";
+import { devServerConfig } from "./dev-server-config";
 
 export type ViteBkndConfig<Env = any> = RuntimeBkndConfig<Env> & {
+   mode?: "cached" | "fresh";
    setAdminHtml?: boolean;
-   forceDev?: boolean;
+   forceDev?: boolean | { mainPath: string };
    html?: string;
 };
 
@@ -24,20 +28,27 @@ ${addBkndContext ? "<!-- BKND_CONTEXT -->" : ""}
    );
 }
 
-async function createApp(config: ViteBkndConfig, env?: any) {
+async function createApp(config: ViteBkndConfig = {}, env?: any) {
+   registerLocalMediaAdapter();
    return await createRuntimeApp(
       {
          ...config,
-         adminOptions: config.setAdminHtml
-            ? { html: config.html, forceDev: config.forceDev }
-            : undefined,
+         adminOptions:
+            config.setAdminHtml === false
+               ? undefined
+               : {
+                    html: config.html,
+                    forceDev: config.forceDev ?? {
+                       mainPath: "/src/main.tsx"
+                    }
+                 },
          serveStatic: ["/assets/*", serveStatic({ root: config.distPath ?? "./" })]
       },
       env
    );
 }
 
-export async function serveFresh(config: ViteBkndConfig) {
+export function serveFresh(config: Omit<ViteBkndConfig, "mode"> = {}) {
    return {
       async fetch(request: Request, env: any, ctx: ExecutionContext) {
          const app = await createApp(config, env);
@@ -47,7 +58,7 @@ export async function serveFresh(config: ViteBkndConfig) {
 }
 
 let app: App;
-export async function serveCached(config: ViteBkndConfig) {
+export function serveCached(config: Omit<ViteBkndConfig, "mode"> = {}) {
    return {
       async fetch(request: Request, env: any, ctx: ExecutionContext) {
          if (!app) {
@@ -58,3 +69,14 @@ export async function serveCached(config: ViteBkndConfig) {
       }
    };
 }
+
+export function serve({ mode, ...config }: ViteBkndConfig = {}) {
+   return mode === "fresh" ? serveFresh(config) : serveCached(config);
+}
+
+export function devServer(options: DevServerOptions) {
+   return honoViteDevServer({
+      ...devServerConfig,
+      ...options
+   });
+}

app/src/auth/AppAuth.ts

@@ -1,9 +1,16 @@
-import { type AuthAction, Authenticator, type ProfileExchange, Role, type Strategy } from "auth";
+import {
+   type AuthAction,
+   AuthPermissions,
+   Authenticator,
+   type ProfileExchange,
+   Role,
+   type Strategy
+} from "auth";
 import type { PasswordStrategy } from "auth/authenticate/strategies";
-import { Exception, type PrimaryFieldType } from "core";
+import { type DB, Exception, type PrimaryFieldType } from "core";
 import { type Static, secureRandomString, transformObject } from "core/utils";
-import { type Entity, EntityIndex, type EntityManager } from "data";
-import { type FieldSchema, entity, enumm, make, text } from "data/prototype";
+import type { Entity, EntityManager } from "data";
+import { type FieldSchema, em, entity, enumm, text } from "data/prototype";
 import { pick } from "lodash-es";
 import { Module } from "modules/Module";
 import { AuthController } from "./api/AuthController";
@@ -17,6 +24,7 @@ declare module "core" {
 }
 
 type AuthSchema = Static<typeof authConfigSchema>;
+export type CreateUserPayload = { email: string; password: string; [key: string]: any };
 
 export class AppAuth extends Module<typeof authConfigSchema> {
    private _authenticator?: Authenticator;
@@ -36,8 +44,12 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       return to;
    }
 
+   get enabled() {
+      return this.config.enabled;
+   }
+
    override async build() {
-      if (!this.config.enabled) {
+      if (!this.enabled) {
          this.setBuilt();
          return;
       }
@@ -72,8 +84,8 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       super.setBuilt();
 
       this._controller = new AuthController(this);
-      //this.ctx.server.use(controller.getMiddleware);
       this.ctx.server.route(this.config.basepath, this._controller.getController());
+      this.ctx.guard.registerPermissions(Object.values(AuthPermissions));
    }
 
    get controller(): AuthController {
@@ -84,14 +96,6 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       return this._controller;
    }
 
-   getMiddleware() {
-      if (!this.config.enabled) {
-         return;
-      }
-
-      return new AuthController(this).getMiddleware;
-   }
-
    getSchema() {
       return authConfigSchema;
    }
@@ -111,12 +115,12 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       identifier: string,
       profile: ProfileExchange
    ): Promise<any> {
-      console.log("***** AppAuth:resolveUser", {
+      /*console.log("***** AppAuth:resolveUser", {
          action,
          strategy: strategy.getName(),
          identifier,
          profile
-      });
+      });*/
       if (!this.config.allow_register && action === "register") {
          throw new Exception("Registration is not allowed", 403);
       }
@@ -137,12 +141,12 @@ export class AppAuth extends Module<typeof authConfigSchema> {
    }
 
    private filterUserData(user: any) {
-      console.log(
+      /*console.log(
          "--filterUserData",
          user,
          this.config.jwt.fields,
          pick(user, this.config.jwt.fields)
-      );
+      );*/
       return pick(user, this.config.jwt.fields);
    }
 
@@ -168,18 +172,18 @@ export class AppAuth extends Module<typeof authConfigSchema> {
       if (!result.data) {
          throw new Exception("User not found", 404);
       }
-      console.log("---login data", result.data, result);
+      //console.log("---login data", result.data, result);
 
       // compare strategy and identifier
-      console.log("strategy comparison", result.data.strategy, strategy.getName());
+      //console.log("strategy comparison", result.data.strategy, strategy.getName());
       if (result.data.strategy !== strategy.getName()) {
-         console.log("!!! User registered with different strategy");
+         //console.log("!!! User registered with different strategy");
          throw new Exception("User registered with different strategy");
       }
 
-      console.log("identifier comparison", result.data.strategy_value, identifier);
+      //console.log("identifier comparison", result.data.strategy_value, identifier);
       if (result.data.strategy_value !== identifier) {
-         console.log("!!! Invalid credentials");
+         //console.log("!!! Invalid credentials");
          throw new Exception("Invalid credentials");
       }
 
@@ -220,10 +224,23 @@ export class AppAuth extends Module<typeof authConfigSchema> {
    }
 
    private toggleStrategyValueVisibility(visible: boolean) {
-      const field = this.getUsersEntity().field("strategy_value")!;
+      const toggle = (name: string, visible: boolean) => {
+         const field = this.getUsersEntity().field(name)!;
+
+         if (visible) {
+            field.config.hidden = false;
+            field.config.fillable = true;
+         } else {
+            // reset to normal
+            const template = AppAuth.usersFields.strategy_value.config;
+            field.config.hidden = template.hidden;
+            field.config.fillable = template.fillable;
+         }
+      };
+
+      toggle("strategy_value", visible);
+      toggle("strategy", visible);
 
-      field.config.hidden = !visible;
-      field.config.fillable = visible;
       // @todo: think about a PasswordField that automatically hashes on save?
    }
 
@@ -238,7 +255,10 @@ export class AppAuth extends Module<typeof authConfigSchema> {
 
    static usersFields = {
       email: text().required(),
-      strategy: text({ fillable: ["create"], hidden: ["form"] }).required(),
+      strategy: text({
+         fillable: ["create"],
+         hidden: ["update", "form"]
+      }).required(),
       strategy_value: text({
          fillable: ["create"],
          hidden: ["read", "table", "update", "form"]
@@ -247,51 +267,34 @@ export class AppAuth extends Module<typeof authConfigSchema> {
    };
 
    registerEntities() {
-      const users = this.getUsersEntity();
-
-      if (!this.em.hasEntity(users.name)) {
-         this.em.addEntity(users);
-      } else {
-         // if exists, check all fields required are there
-         // @todo: add to context: "needs sync" flag
-         const _entity = this.getUsersEntity(true);
-         for (const field of _entity.fields) {
-            const _field = users.field(field.name);
-            if (!_field) {
-               users.addField(field);
+      const users = this.getUsersEntity(true);
+      this.ensureSchema(
+         em(
+            {
+               [users.name as "users"]: users
+            },
+            ({ index }, { users }) => {
+               index(users).on(["email"], true).on(["strategy"]).on(["strategy_value"]);
             }
-         }
-      }
-
-      const indices = [
-         new EntityIndex(users, [users.field("email")!], true),
-         new EntityIndex(users, [users.field("strategy")!]),
-         new EntityIndex(users, [users.field("strategy_value")!])
-      ];
-      indices.forEach((index) => {
-         if (!this.em.hasIndex(index)) {
-            this.em.addIndex(index);
-         }
-      });
+         )
+      );
 
       try {
          const roles = Object.keys(this.config.roles ?? {});
-         const field = make("role", enumm({ enum: roles }));
-         this.em.entity(users.name).__experimental_replaceField("role", field);
+         this.replaceEntityField(users, "role", enumm({ enum: roles }));
       } catch (e) {}
 
       try {
          const strategies = Object.keys(this.config.strategies ?? {});
-         const field = make("strategy", enumm({ enum: strategies }));
-         this.em.entity(users.name).__experimental_replaceField("strategy", field);
+         this.replaceEntityField(users, "strategy", enumm({ enum: strategies }));
       } catch (e) {}
    }
 
-   async createUser({
-      email,
-      password,
-      ...additional
-   }: { email: string; password: string; [key: string]: any }) {
+   async createUser({ email, password, ...additional }: CreateUserPayload): Promise<DB["users"]> {
+      if (!this.enabled) {
+         throw new Error("Cannot create user, auth not enabled");
+      }
+
       const strategy = "password";
       const pw = this.authenticator.strategy(strategy) as PasswordStrategy;
       const strategy_value = await pw.hash(password);

app/src/auth/api/AuthApi.ts

@@ -1,4 +1,5 @@
-import type { AppAuthSchema, AppAuthStrategies } from "auth/auth-schema";
+import type { AuthActionResponse } from "auth/api/AuthController";
+import type { AppAuthSchema } from "auth/auth-schema";
 import type { AuthResponse, SafeUser, Strategy } from "auth/authenticate/Authenticator";
 import { type BaseModuleApiOptions, ModuleApi } from "modules/ModuleApi";
 
@@ -13,22 +14,52 @@ export class AuthApi extends ModuleApi<AuthApiOptions> {
       };
    }
 
-   async loginWithPassword(input: any) {
-      const res = await this.post<AuthResponse>(["password", "login"], input);
+   async login(strategy: string, input: any) {
+      const res = await this.post<AuthResponse>([strategy, "login"], input, {
+         credentials: "include"
+      });
+
       if (res.ok && res.body.token) {
          await this.options.onTokenUpdate?.(res.body.token);
       }
       return res;
    }
 
-   async registerWithPassword(input: any) {
-      const res = await this.post<AuthResponse>(["password", "register"], input);
+   async register(strategy: string, input: any) {
+      const res = await this.post<AuthResponse>([strategy, "register"], input, {
+         credentials: "include"
+      });
+
       if (res.ok && res.body.token) {
          await this.options.onTokenUpdate?.(res.body.token);
       }
       return res;
    }
 
+   async actionSchema(strategy: string, action: string) {
+      return this.get<Strategy>([strategy, "actions", action, "schema.json"]);
+   }
+
+   async action(strategy: string, action: string, input: any) {
+      return this.post<AuthActionResponse>([strategy, "actions", action], input);
+   }
+
+   /**
+    * @deprecated use login("password", ...) instead
+    * @param input
+    */
+   async loginWithPassword(input: any) {
+      return this.login("password", input);
+   }
+
+   /**
+    * @deprecated use register("password", ...) instead
+    * @param input
+    */
+   async registerWithPassword(input: any) {
+      return this.register("password", input);
+   }
+
    me() {
       return this.get<{ user: SafeUser | null }>(["me"]);
    }

app/src/auth/api/AuthController.ts

@@ -1,58 +1,120 @@
-import type { AppAuth } from "auth";
-import { type ClassController, isDebug } from "core";
-import { Hono, type MiddlewareHandler } from "hono";
+import { type AppAuth, AuthPermissions, type SafeUser, type Strategy } from "auth";
+import { TypeInvalidError, parse } from "core/utils";
+import { DataPermissions } from "data";
+import type { Hono } from "hono";
+import { Controller } from "modules/Controller";
+import type { ServerEnv } from "modules/Module";
 
-export class AuthController implements ClassController {
-   constructor(private auth: AppAuth) {}
+export type AuthActionResponse = {
+   success: boolean;
+   action: string;
+   data?: SafeUser;
+   errors?: any;
+};
+
+export class AuthController extends Controller {
+   constructor(private auth: AppAuth) {
+      super();
+   }
 
    get guard() {
       return this.auth.ctx.guard;
    }
 
-   getMiddleware: MiddlewareHandler = async (c, next) => {
-      // @todo: ONLY HOTFIX
-      // middlewares are added for all routes are registered. But we need to make sure that
-      // only HTML/JSON routes are adding a cookie to the response. Config updates might
-      // also use an extension "syntax", e.g. /api/system/patch/data/entities.posts
-      // This middleware should be extracted and added by each Controller individually,
-      // but it requires access to the auth secret.
-      // Note: This doesn't mean endpoints aren't protected, just the cookie is not set.
-      const url = new URL(c.req.url);
-      const last = url.pathname.split("/")?.pop();
-      const ext = last?.includes(".") ? last.split(".")?.pop() : undefined;
-      if (
-         !this.auth.authenticator.isJsonRequest(c) &&
-         ["GET", "HEAD", "OPTIONS"].includes(c.req.method) &&
-         ext &&
-         ["js", "css", "png", "jpg", "jpeg", "svg", "ico"].includes(ext)
-      ) {
-         isDebug() && console.log("Skipping auth", { ext }, url.pathname);
-      } else {
-         const user = await this.auth.authenticator.resolveAuthFromRequest(c);
-         this.auth.ctx.guard.setUserContext(user);
+   get em() {
+      return this.auth.ctx.em;
+   }
+
+   get userRepo() {
+      const entity_name = this.auth.config.entity_name;
+      return this.em.repo(entity_name as "users");
+   }
+
+   private registerStrategyActions(strategy: Strategy, mainHono: Hono<ServerEnv>) {
+      const actions = strategy.getActions?.();
+      if (!actions) {
+         return;
       }
 
-      await next();
-   };
+      const { auth, permission } = this.middlewares;
+      const hono = this.create().use(auth());
 
-   getController(): Hono<any> {
-      const hono = new Hono();
+      const name = strategy.getName();
+      const { create, change } = actions;
+      const em = this.auth.em;
+
+      if (create) {
+         hono.post(
+            "/create",
+            permission([AuthPermissions.createUser, DataPermissions.entityCreate]),
+            async (c) => {
+               try {
+                  const body = await this.auth.authenticator.getBody(c);
+                  const valid = parse(create.schema, body, {
+                     skipMark: true
+                  });
+                  const processed = (await create.preprocess?.(valid)) ?? valid;
+
+                  // @todo: check processed for "role" and check permissions
+                  const mutator = em.mutator(this.auth.config.entity_name as "users");
+                  mutator.__unstable_toggleSystemEntityCreation(false);
+                  const { data: created } = await mutator.insertOne({
+                     ...processed,
+                     strategy: name
+                  });
+                  mutator.__unstable_toggleSystemEntityCreation(true);
+
+                  return c.json({
+                     success: true,
+                     action: "create",
+                     strategy: name,
+                     data: created as unknown as SafeUser
+                  } as AuthActionResponse);
+               } catch (e) {
+                  if (e instanceof TypeInvalidError) {
+                     return c.json(
+                        {
+                           success: false,
+                           errors: e.errors
+                        },
+                        400
+                     );
+                  }
+                  throw e;
+               }
+            }
+         );
+         hono.get("create/schema.json", async (c) => {
+            return c.json(create.schema);
+         });
+      }
+
+      mainHono.route(`/${name}/actions`, hono);
+   }
+
+   override getController() {
+      const { auth } = this.middlewares;
+      const hono = this.create();
       const strategies = this.auth.authenticator.getStrategies();
 
       for (const [name, strategy] of Object.entries(strategies)) {
          //console.log("registering", name, "at", `/${name}`);
          hono.route(`/${name}`, strategy.getController(this.auth.authenticator));
+         this.registerStrategyActions(strategy, hono);
       }
 
-      hono.get("/me", async (c) => {
+      hono.get("/me", auth(), async (c) => {
          if (this.auth.authenticator.isUserLoggedIn()) {
-            return c.json({ user: await this.auth.authenticator.getUser() });
+            const claims = this.auth.authenticator.getUser()!;
+            const { data: user } = await this.userRepo.findId(claims.id);
+
+            return c.json({ user });
          }
 
          return c.json({ user: null }, 403);
       });
 
-      hono.get("/logout", async (c) => {
+      hono.get("/logout", auth(), async (c) => {
          await this.auth.authenticator.logout(c);
          if (this.auth.authenticator.isJsonRequest(c)) {
             return c.json({ ok: true });

app/src/auth/auth-permissions.ts

@@ -0,0 +1,4 @@
+import { Permission } from "core";
+
+export const createUser = new Permission("auth.user.create");
+//export const updateUser = new Permission("auth.user.update");

app/src/auth/auth-schema.ts

@@ -33,6 +33,7 @@ const strategiesSchemaObject = objectTransform(STRATEGIES, (strategy, name) => {
 const strategiesSchema = Type.Union(Object.values(strategiesSchemaObject));
 export type AppAuthStrategies = Static<typeof strategiesSchema>;
 export type AppAuthOAuthStrategy = Static<typeof STRATEGIES.oauth.schema>;
+export type AppAuthCustomOAuthStrategy = Static<typeof STRATEGIES.custom_oauth.schema>;
 
 const guardConfigSchema = Type.Object({
    enabled: Type.Optional(Type.Boolean({ default: false }))

app/src/auth/authenticate/Authenticator.ts

@@ -1,23 +1,31 @@
-import { Exception } from "core";
+import { type DB, Exception } from "core";
 import { addFlashMessage } from "core/server/flash";
 import {
    type Static,
    StringEnum,
-   type TSchema,
+   type TObject,
    Type,
    parse,
-   randomString,
+   runtimeSupports,
    transformObject
 } from "core/utils";
 import type { Context, Hono } from "hono";
 import { deleteCookie, getSignedCookie, setSignedCookie } from "hono/cookie";
 import { sign, verify } from "hono/jwt";
 import type { CookieOptions } from "hono/utils/cookie";
-import { omit } from "lodash-es";
+import type { ServerEnv } from "modules/Module";
 
 type Input = any; // workaround
 export type JWTPayload = Parameters<typeof sign>[0];
 
+export const strategyActions = ["create", "change"] as const;
+export type StrategyActionName = (typeof strategyActions)[number];
+export type StrategyAction<S extends TObject = TObject> = {
+   schema: S;
+   preprocess: (input: unknown) => Promise<Omit<DB["users"], "id" | "strategy">>;
+};
+export type StrategyActions = Partial<Record<StrategyActionName, StrategyAction>>;
+
 // @todo: add schema to interface to ensure proper inference
 export interface Strategy {
    getController: (auth: Authenticator) => Hono<any>;
@@ -25,6 +33,7 @@ export interface Strategy {
    getMode: () => "form" | "external";
    getName: () => string;
    toJSON: (secrets?: boolean) => any;
+   getActions?: () => StrategyActions;
 }
 
 export type User = {
@@ -67,6 +76,9 @@ export const cookieConfig = Type.Partial(
    { default: {}, additionalProperties: false }
 );
 
+// @todo: maybe add a config to not allow cookie/api tokens to be used interchangably?
+// see auth.integration test for further details
+
 export const jwtConfig = Type.Object(
    {
       // @todo: autogenerate a secret if not present. But it must be persisted from AppAuth
@@ -98,7 +110,13 @@ export type AuthUserResolver = (
 export class Authenticator<Strategies extends Record<string, Strategy> = Record<string, Strategy>> {
    private readonly strategies: Strategies;
    private readonly config: AuthConfig;
-   private _user: SafeUser | undefined;
+   private _claims:
+      | undefined
+      | (SafeUser & {
+           iat: number;
+           iss?: string;
+           exp?: number;
+        });
    private readonly userResolver: AuthUserResolver;
 
    constructor(strategies: Strategies, userResolver?: AuthUserResolver, config?: AuthConfig) {
@@ -131,16 +149,18 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
    }
 
    isUserLoggedIn(): boolean {
-      return this._user !== undefined;
+      return this._claims !== undefined;
    }
 
-   getUser() {
-      return this._user;
+   getUser(): SafeUser | undefined {
+      if (!this._claims) return;
+
+      const { iat, exp, iss, ...user } = this._claims;
+      return user;
    }
 
-   // @todo: determine what to do exactly
-   __setUserNull() {
-      this._user = undefined;
+   resetUser() {
+      this._claims = undefined;
    }
 
    strategy<
@@ -154,6 +174,7 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       }
    }
 
+   // @todo: add jwt tests
    async jwt(user: Omit<User, "password">): Promise<string> {
       const prohibited = ["password"];
       for (const prop of prohibited) {
@@ -200,11 +221,11 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
             }
          }
 
-         this._user = omit(payload, ["iat", "exp", "iss"]) as SafeUser;
+         this._claims = payload as any;
          return true;
       } catch (e) {
-         this._user = undefined;
-         console.error(e);
+         this.resetUser();
+         //console.error(e);
       }
 
       return false;
@@ -222,10 +243,8 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
    private async getAuthCookie(c: Context): Promise<string | undefined> {
       try {
          const secret = this.config.jwt.secret;
-
          const token = await getSignedCookie(c, secret, "auth");
          if (typeof token !== "string") {
-            await deleteCookie(c, "auth", this.cookieOptions);
             return undefined;
          }
 
@@ -240,26 +259,30 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
    }
 
    async requestCookieRefresh(c: Context) {
-      if (this.config.cookie.renew) {
+      if (this.config.cookie.renew && this.isUserLoggedIn()) {
          const token = await this.getAuthCookie(c);
          if (token) {
-            console.log("renewing cookie", c.req.url);
             await this.setAuthCookie(c, token);
          }
       }
    }
 
-   private async setAuthCookie(c: Context, token: string) {
+   private async setAuthCookie(c: Context<ServerEnv>, token: string) {
       const secret = this.config.jwt.secret;
       await setSignedCookie(c, "auth", token, secret, this.cookieOptions);
    }
 
+   private async deleteAuthCookie(c: Context) {
+      await deleteCookie(c, "auth", this.cookieOptions);
+   }
+
    async logout(c: Context) {
       const cookie = await this.getAuthCookie(c);
       if (cookie) {
-         await deleteCookie(c, "auth", this.cookieOptions);
+         await this.deleteAuthCookie(c);
          await addFlashMessage(c, "Signed out", "info");
       }
+      this.resetUser();
    }
 
    // @todo: move this to a server helper
@@ -268,27 +291,54 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       return c.req.header("Content-Type") === "application/json";
    }
 
-   async respond(c: Context, data: AuthResponse | Error | any, redirect?: string) {
+   async getBody(c: Context) {
       if (this.isJsonRequest(c)) {
-         return c.json(data);
+         return await c.req.json();
+      } else {
+         return Object.fromEntries((await c.req.formData()).entries());
+      }
+   }
+
+   private getSafeUrl(c: Context, path: string) {
+      const p = path.replace(/\/+$/, "/");
+
+      // nextjs doesn't support non-fq urls
+      // but env could be proxied (stackblitz), so we shouldn't fq every url
+      if (!runtimeSupports("redirects_non_fq")) {
+         return new URL(c.req.url).origin + p;
       }
 
-      const successPath = this.config.cookie.pathSuccess ?? "/";
-      const successUrl = new URL(c.req.url).origin + successPath.replace(/\/+$/, "/");
-      const referer = new URL(redirect ?? c.req.header("Referer") ?? successUrl);
+      return p;
+   }
+
+   async respond(c: Context, data: AuthResponse | Error | any, redirect?: string) {
+      const successUrl = this.getSafeUrl(c, redirect ?? this.config.cookie.pathSuccess ?? "/");
+      const referer = redirect ?? c.req.header("Referer") ?? successUrl;
+      //console.log("auth respond", { redirect, successUrl, successPath });
 
       if ("token" in data) {
          await this.setAuthCookie(c, data.token);
+
+         if (this.isJsonRequest(c)) {
+            return c.json(data);
+         }
+
          // can't navigate to "/" – doesn't work on nextjs
+         //console.log("auth success, redirecting to", successUrl);
          return c.redirect(successUrl);
       }
 
+      if (this.isJsonRequest(c)) {
+         return c.json(data, 400);
+      }
+
       let message = "An error occured";
       if (data instanceof Exception) {
          message = data.message;
       }
 
       await addFlashMessage(c, message, "error");
+      //console.log("auth failed, redirecting to", referer);
       return c.redirect(referer);
    }
 
@@ -304,7 +354,7 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
 
       if (token) {
          await this.verify(token);
-         return this._user;
+         return this.getUser();
       }
 
       return undefined;
@@ -318,3 +368,13 @@ export class Authenticator<Strategies extends Record<string, Strategy> = Record<
       };
    }
 }
+
+export function createStrategyAction<S extends TObject>(
+   schema: S,
+   preprocess: (input: Static<S>) => Promise<Partial<DB["users"]>>
+) {
+   return {
+      schema,
+      preprocess
+   } as StrategyAction<S>;
+}

app/src/auth/authenticate/strategies/PasswordStrategy.ts

@@ -1,7 +1,9 @@
 import type { Authenticator, Strategy } from "auth";
+import { isDebug, tbValidator as tb } from "core";
 import { type Static, StringEnum, Type, parse } from "core/utils";
 import { hash } from "core/utils";
 import { type Context, Hono } from "hono";
+import { type StrategyAction, type StrategyActions, createStrategyAction } from "../Authenticator";
 
 type LoginSchema = { username: string; password: string } | { email: string; password: string };
 type RegisterSchema = { email: string; password: string; [key: string]: any };
@@ -54,35 +56,78 @@ export class PasswordStrategy implements Strategy {
    getController(authenticator: Authenticator): Hono<any> {
       const hono = new Hono();
 
-      async function getBody(c: Context) {
-         if (authenticator.isJsonRequest(c)) {
-            return await c.req.json();
-         } else {
-            return Object.fromEntries((await c.req.formData()).entries());
-         }
-      }
-
       return hono
-         .post("/login", async (c) => {
-            const body = await getBody(c);
+         .post(
+            "/login",
+            tb(
+               "query",
+               Type.Object({
+                  redirect: Type.Optional(Type.String())
+               })
+            ),
+            async (c) => {
+               const body = await authenticator.getBody(c);
+               const { redirect } = c.req.valid("query");
 
-            try {
-               const payload = await this.login(body);
-               const data = await authenticator.resolve("login", this, payload.password, payload);
+               try {
+                  const payload = await this.login(body);
+                  const data = await authenticator.resolve(
+                     "login",
+                     this,
+                     payload.password,
+                     payload
+                  );
 
-               return await authenticator.respond(c, data);
-            } catch (e) {
-               return await authenticator.respond(c, e);
+                  return await authenticator.respond(c, data, redirect);
+               } catch (e) {
+                  return await authenticator.respond(c, e);
+               }
             }
-         })
-         .post("/register", async (c) => {
-            const body = await getBody(c);
+         )
+         .post(
+            "/register",
+            tb(
+               "query",
+               Type.Object({
+                  redirect: Type.Optional(Type.String())
+               })
+            ),
+            async (c) => {
+               const body = await authenticator.getBody(c);
+               const { redirect } = c.req.valid("query");
 
-            const payload = await this.register(body);
-            const data = await authenticator.resolve("register", this, payload.password, payload);
+               const payload = await this.register(body);
+               const data = await authenticator.resolve(
+                  "register",
+                  this,
+                  payload.password,
+                  payload
+               );
 
-            return await authenticator.respond(c, data);
-         });
+               return await authenticator.respond(c, data, redirect);
+            }
+         );
+   }
+
+   getActions(): StrategyActions {
+      return {
+         create: createStrategyAction(
+            Type.Object({
+               email: Type.String({
+                  pattern: "^[\\w-\\.]+@([\\w-]+\\.)+[\\w-]{2,4}$"
+               }),
+               password: Type.String({
+                  minLength: 8 // @todo: this should be configurable
+               })
+            }),
+            async ({ password, ...input }) => {
+               return {
+                  ...input,
+                  strategy_value: await this.hash(password)
+               };
+            }
+         )
+      };
    }
 
    getSchema() {

app/src/auth/authorize/Guard.ts

@@ -98,12 +98,16 @@ export class Guard {
       if (this.user && typeof this.user.role === "string") {
          const role = this.roles?.find((role) => role.name === this.user?.role);
          if (role) {
-            debug && console.log("guard: role found", this.user.role);
+            debug && console.log("guard: role found", [this.user.role]);
             return role;
          }
       }
 
-      debug && console.log("guard: role not found", this.user, this.user?.role);
+      debug &&
+         console.log("guard: role not found", {
+            user: this.user,
+            role: this.user?.role
+         });
       return this.getDefaultRole();
    }
 

app/src/auth/index.ts

@@ -19,3 +19,5 @@ export { AppAuth, type UserFieldSchema } from "./AppAuth";
 
 export { Guard, type GuardUserContext, type GuardConfig } from "./authorize/Guard";
 export { Role } from "./authorize/Role";
+
+export * as AuthPermissions from "./auth-permissions";

app/src/auth/middlewares.ts

@@ -0,0 +1,108 @@
+import type { Permission } from "core";
+import { patternMatch } from "core/utils";
+import type { Context } from "hono";
+import { createMiddleware } from "hono/factory";
+import type { ServerEnv } from "modules/Module";
+
+function getPath(reqOrCtx: Request | Context) {
+   const req = reqOrCtx instanceof Request ? reqOrCtx : reqOrCtx.req.raw;
+   return new URL(req.url).pathname;
+}
+
+export function shouldSkip(c: Context<ServerEnv>, skip?: (string | RegExp)[]) {
+   if (c.get("auth_skip")) return true;
+
+   const req = c.req.raw;
+   if (!skip) return false;
+
+   const path = getPath(req);
+   const result = skip.some((s) => patternMatch(path, s));
+
+   c.set("auth_skip", result);
+   return result;
+}
+
+export const auth = (options?: {
+   skip?: (string | RegExp)[];
+}) =>
+   createMiddleware<ServerEnv>(async (c, next) => {
+      const app = c.get("app");
+      const guard = app?.modules.ctx().guard;
+      const authenticator = app?.module.auth.authenticator;
+
+      let skipped = shouldSkip(c, options?.skip) || !app?.module.auth.enabled;
+
+      // make sure to only register once
+      if (c.get("auth_registered")) {
+         skipped = true;
+         console.warn(`auth middleware already registered for ${getPath(c)}`);
+      } else {
+         c.set("auth_registered", true);
+
+         if (!skipped) {
+            const resolved = c.get("auth_resolved");
+            if (!resolved) {
+               if (!app?.module.auth.enabled) {
+                  guard?.setUserContext(undefined);
+               } else {
+                  guard?.setUserContext(await authenticator?.resolveAuthFromRequest(c));
+                  c.set("auth_resolved", true);
+               }
+            }
+         }
+      }
+
+      await next();
+
+      if (!skipped) {
+         // renew cookie if applicable
+         authenticator?.requestCookieRefresh(c);
+      }
+
+      // release
+      guard?.setUserContext(undefined);
+      authenticator?.resetUser();
+      c.set("auth_resolved", false);
+   });
+
+export const permission = (
+   permission: Permission | Permission[],
+   options?: {
+      onGranted?: (c: Context<ServerEnv>) => Promise<Response | void | undefined>;
+      onDenied?: (c: Context<ServerEnv>) => Promise<Response | void | undefined>;
+   }
+) =>
+   // @ts-ignore
+   createMiddleware<ServerEnv>(async (c, next) => {
+      const app = c.get("app");
+      //console.log("skip?", c.get("auth_skip"));
+
+      // in tests, app is not defined
+      if (!c.get("auth_registered") || !app) {
+         const msg = `auth middleware not registered, cannot check permissions for ${getPath(c)}`;
+         if (app?.module.auth.enabled) {
+            throw new Error(msg);
+         } else {
+            console.warn(msg);
+         }
+      } else if (!c.get("auth_skip")) {
+         const guard = app.modules.ctx().guard;
+         const permissions = Array.isArray(permission) ? permission : [permission];
+
+         if (options?.onGranted || options?.onDenied) {
+            let returned: undefined | void | Response;
+            if (permissions.every((p) => guard.granted(p))) {
+               returned = await options?.onGranted?.(c);
+            } else {
+               returned = await options?.onDenied?.(c);
+            }
+            if (returned instanceof Response) {
+               return returned;
+            }
+         } else {
+            permissions.some((p) => guard.throwUnlessGranted(p));
+         }
+      }
+
+      await next();
+   });

app/src/cli/commands/run/platform.ts

@@ -1,5 +1,6 @@
 import path from "node:path";
 import type { Config } from "@libsql/client/node";
+import { config } from "core";
 import type { MiddlewareHandler } from "hono";
 import open from "open";
 import { fileExists, getRelativeDistPath } from "../../utils/sys";
@@ -26,7 +27,7 @@ export async function serveStatic(server: Platform): Promise<MiddlewareHandler>
 }
 
 export async function attachServeStatic(app: any, platform: Platform) {
-   app.module.server.client.get("/*", await serveStatic(platform));
+   app.module.server.client.get(config.server.assets_path + "*", await serveStatic(platform));
 }
 
 export async function startServer(server: Platform, app: any, options: { port: number }) {

app/src/cli/commands/user.ts

@@ -35,9 +35,11 @@ async function action(action: "create" | "update", options: any) {
 }
 
 async function create(app: App, options: any) {
-   const config = app.module.auth.toJSON(true);
    const strategy = app.module.auth.authenticator.strategy("password") as PasswordStrategy;
-   const users_entity = config.entity_name as "users";
+
+   if (!strategy) {
+      throw new Error("Password strategy not configured");
+   }
 
    const email = await $text({
       message: "Enter email",
@@ -65,16 +67,11 @@ async function create(app: App, options: any) {
    }
 
    try {
-      const mutator = app.modules.ctx().em.mutator(users_entity);
-      mutator.__unstable_toggleSystemEntityCreation(false);
-      const res = await mutator.insertOne({
+      const created = await app.createUser({
          email,
-         strategy: "password",
-         strategy_value: await strategy.hash(password as string)
-      });
-      mutator.__unstable_toggleSystemEntityCreation(true);
-
-      console.log("Created:", res.data);
+         password: await strategy.hash(password as string)
+      })
+      console.log("Created:", created);
    } catch (e) {
       console.error("Error", e);
    }
@@ -141,4 +138,4 @@ async function update(app: App, options: any) {
    } catch (e) {
       console.error("Error", e);
    }
-}
+}

app/src/core/config.ts

@@ -5,12 +5,19 @@ import type { Generated } from "kysely";
 
 export type PrimaryFieldType = number | Generated<number>;
 
-// biome-ignore lint/suspicious/noEmptyInterface: <explanation>
-export interface DB {}
+export interface DB {
+   // make sure to make unknown as "any"
+   [key: string]: {
+      id: PrimaryFieldType;
+      [key: string]: any;
+   };
+}
 
 export const config = {
    server: {
-      default_port: 1337
+      default_port: 1337,
+      // resetted to root for now, bc bundling with vite
+      assets_path: "/"
    },
    data: {
       default_primary_field: "id"

app/src/core/errors.ts

@@ -1,6 +1,7 @@
 export class Exception extends Error {
    code = 400;
    override name = "Exception";
+   protected _context = undefined;
 
    constructor(message: string, code?: number) {
       super(message);
@@ -9,11 +10,16 @@ export class Exception extends Error {
       }
    }
 
+   context(context: any) {
+      this._context = context;
+      return this;
+   }
+
    toJSON() {
       return {
          error: this.message,
-         type: this.name
-         //message: this.message
+         type: this.name,
+         context: this._context
       };
    }
 }

app/src/core/events/Event.ts

@@ -1,17 +1,38 @@
-export abstract class Event<Params = any> {
+export type EventClass = {
+   new (params: any): Event<any, any>;
+   slug: string;
+};
+
+export abstract class Event<Params = any, Returning = void> {
+   _returning!: Returning;
+
    /**
     * Unique event slug
     * Must be static, because registering events is done by class
     */
    static slug: string = "untitled-event";
    params: Params;
+   returned: boolean = false;
+
+   validate(value: Returning): Event<Params, Returning> | void {
+      throw new EventReturnedWithoutValidation(this as any, value);
+   }
+
+   protected clone<This extends Event<Params, Returning> = Event<Params, Returning>>(
+      this: This,
+      params: Params
+   ): This {
+      const cloned = new (this.constructor as any)(params);
+      cloned.returned = true;
+      return cloned as This;
+   }
 
    constructor(params: Params) {
       this.params = params;
    }
 }
 
-// @todo: current workaround: potentially there is none and that's the way
+// @todo: current workaround: potentially there is "none" and that's the way
 export class NoParamEvent extends Event<null> {
    static override slug: string = "noparam-event";
 
@@ -19,3 +40,19 @@ export class NoParamEvent extends Event<null> {
       super(null);
    }
 }
+
+export class InvalidEventReturn extends Error {
+   constructor(expected: string, given: string) {
+      super(`Expected "${expected}", got "${given}"`);
+   }
+}
+
+export class EventReturnedWithoutValidation extends Error {
+   constructor(
+      event: EventClass,
+      public data: any
+   ) {
+      // @ts-expect-error slug is static
+      super(`Event "${event.constructor.slug}" returned without validation`);
+   }
+}

app/src/core/events/EventListener.ts

@@ -4,15 +4,16 @@ import type { EventClass } from "./EventManager";
 export const ListenerModes = ["sync", "async"] as const;
 export type ListenerMode = (typeof ListenerModes)[number];
 
-export type ListenerHandler<E extends Event = Event> = (
+export type ListenerHandler<E extends Event<any, any>> = (
    event: E,
-   slug: string,
-) => Promise<void> | void;
+   slug: string
+) => E extends Event<any, infer R> ? R | Promise<R | void> : never;
 
 export class EventListener<E extends Event = Event> {
    mode: ListenerMode = "async";
    event: EventClass;
    handler: ListenerHandler<E>;
+   once: boolean = false;
 
    constructor(event: EventClass, handler: ListenerHandler<E>, mode: ListenerMode = "async") {
       this.event = event;

app/src/core/events/EventManager.ts

@@ -1,14 +1,19 @@
-import type { Event } from "./Event";
+import { type Event, type EventClass, InvalidEventReturn } from "./Event";
 import { EventListener, type ListenerHandler, type ListenerMode } from "./EventListener";
 
+export type RegisterListenerConfig =
+   | ListenerMode
+   | {
+        mode?: ListenerMode;
+        once?: boolean;
+     };
+
 export interface EmitsEvents {
    emgr: EventManager;
 }
 
-export type EventClass = {
-   new (params: any): Event;
-   slug: string;
-};
+// for compatibility, moved it to Event.ts
+export type { EventClass };
 
 export class EventManager<
    RegisteredEvents extends Record<string, EventClass> = Record<string, EventClass>
@@ -17,16 +22,20 @@ export class EventManager<
    protected listeners: EventListener[] = [];
    enabled: boolean = true;
 
-   constructor(events?: RegisteredEvents, listeners?: EventListener[]) {
+   constructor(
+      events?: RegisteredEvents,
+      private options?: {
+         listeners?: EventListener[];
+         onError?: (event: Event, e: unknown) => void;
+         onInvalidReturn?: (event: Event, e: InvalidEventReturn) => void;
+         asyncExecutor?: typeof Promise.all;
+      }
+   ) {
       if (events) {
          this.registerEvents(events);
       }
 
-      if (listeners) {
-         for (const listener of listeners) {
-            this.addListener(listener);
-         }
-      }
+      options?.listeners?.forEach((l) => this.addListener(l));
    }
 
    enable() {
@@ -82,9 +91,11 @@ export class EventManager<
       return !!this.events.find((e) => slug === e.slug);
    }
 
-   protected throwIfEventNotRegistered(event: EventClass) {
-      if (!this.eventExists(event)) {
-         throw new Error(`Event "${event.slug}" not registered`);
+   protected throwIfEventNotRegistered(event: EventClass | Event | string) {
+      if (!this.eventExists(event as any)) {
+         // @ts-expect-error
+         const name = event.constructor?.slug ?? event.slug ?? event;
+         throw new Error(`Event "${name}" not registered`);
       }
    }
 
@@ -117,55 +128,108 @@ export class EventManager<
       return this;
    }
 
+   protected createEventListener(
+      _event: EventClass | string,
+      handler: ListenerHandler<any>,
+      _config: RegisterListenerConfig = "async"
+   ) {
+      const event =
+         typeof _event === "string" ? this.events.find((e) => e.slug === _event)! : _event;
+      const config = typeof _config === "string" ? { mode: _config } : _config;
+      const listener = new EventListener(event, handler, config.mode);
+      if (config.once) {
+         listener.once = true;
+      }
+      this.addListener(listener as any);
+   }
+
    onEvent<ActualEvent extends EventClass, Instance extends InstanceType<ActualEvent>>(
       event: ActualEvent,
       handler: ListenerHandler<Instance>,
-      mode: ListenerMode = "async"
+      config?: RegisterListenerConfig
    ) {
-      this.throwIfEventNotRegistered(event);
-
-      const listener = new EventListener(event, handler, mode);
-      this.addListener(listener as any);
+      this.createEventListener(event, handler, config);
    }
 
    on<Params = any>(
       slug: string,
       handler: ListenerHandler<Event<Params>>,
-      mode: ListenerMode = "async"
+      config?: RegisterListenerConfig
    ) {
-      const event = this.events.find((e) => e.slug === slug);
-      if (!event) {
-         throw new Error(`Event "${slug}" not registered`);
-      }
-
-      this.onEvent(event, handler, mode);
+      this.createEventListener(slug, handler, config);
    }
 
-   onAny(handler: ListenerHandler<Event<unknown>>, mode: ListenerMode = "async") {
-      this.events.forEach((event) => this.onEvent(event, handler, mode));
+   onAny(handler: ListenerHandler<Event<unknown>>, config?: RegisterListenerConfig) {
+      this.events.forEach((event) => this.onEvent(event, handler, config));
    }
 
-   async emit(event: Event) {
+   protected executeAsyncs(promises: (() => Promise<void>)[]) {
+      const executor = this.options?.asyncExecutor ?? ((e) => Promise.all(e));
+      executor(promises.map((p) => p())).then(() => void 0);
+   }
+
+   async emit<Actual extends Event<any, any>>(event: Actual): Promise<Actual> {
       // @ts-expect-error slug is static
       const slug = event.constructor.slug;
       if (!this.enabled) {
          console.log("EventManager disabled, not emitting", slug);
-         return;
+         return event;
       }
 
       if (!this.eventExists(event)) {
          throw new Error(`Event "${slug}" not registered`);
       }
 
-      const listeners = this.listeners.filter((listener) => listener.event.slug === slug);
-      //console.log("---!-- emitting", slug, listeners.length);
+      const syncs: EventListener[] = [];
+      const asyncs: (() => Promise<void>)[] = [];
+
+      this.listeners = this.listeners.filter((listener) => {
+         // if no match, keep and ignore
+         if (listener.event.slug !== slug) return true;
 
-      for (const listener of listeners) {
          if (listener.mode === "sync") {
-            await listener.handler(event, listener.event.slug);
+            syncs.push(listener);
          } else {
-            listener.handler(event, listener.event.slug);
+            asyncs.push(async () => await listener.handler(event, listener.event.slug));
+         }
+         // Remove if `once` is true, otherwise keep
+         return !listener.once;
+      });
+
+      // execute asyncs
+      this.executeAsyncs(asyncs);
+
+      // execute syncs
+      let _event: Actual = event;
+      for (const listener of syncs) {
+         try {
+            const return_value = (await listener.handler(_event, listener.event.slug)) as any;
+
+            if (typeof return_value !== "undefined") {
+               const newEvent = _event.validate(return_value);
+               // @ts-expect-error slug is static
+               if (newEvent && newEvent.constructor.slug === slug) {
+                  if (!newEvent.returned) {
+                     throw new Error(
+                        // @ts-expect-error slug is static
+                        `Returned event ${newEvent.constructor.slug} must be marked as returned.`
+                     );
+                  }
+                  _event = newEvent as Actual;
+               }
+            }
+         } catch (e) {
+            if (e instanceof InvalidEventReturn) {
+               this.options?.onInvalidReturn?.(_event, e);
+               console.warn(`Invalid return of event listener for "${slug}": ${e.message}`);
+            } else if (this.options?.onError) {
+               this.options.onError(_event, e);
+            } else {
+               throw e;
+            }
          }
       }
+
+      return _event;
    }
 }

app/src/core/events/index.ts

@@ -1,8 +1,8 @@
-export { Event, NoParamEvent } from "./Event";
+export { Event, NoParamEvent, InvalidEventReturn } from "./Event";
 export {
    EventListener,
    ListenerModes,
    type ListenerMode,
-   type ListenerHandler,
+   type ListenerHandler
 } from "./EventListener";
 export { EventManager, type EmitsEvents, type EventClass } from "./EventManager";

app/src/core/object/SchemaObject.ts

@@ -1,10 +1,10 @@
-import { cloneDeep, get, has, mergeWith, omit, set } from "lodash-es";
+import { get, has, omit, set } from "lodash-es";
 import {
    Default,
    type Static,
    type TObject,
    getFullPathKeys,
-   mark,
+   mergeObjectWith,
    parse,
    stripMark
 } from "../utils";
@@ -33,7 +33,7 @@ export class SchemaObject<Schema extends TObject> {
    ) {
       this._default = Default(_schema, {} as any) as any;
       this._value = initial
-         ? parse(_schema, cloneDeep(initial as any), {
+         ? parse(_schema, structuredClone(initial as any), {
               forceParse: this.isForceParse(),
               skipMark: this.isForceParse()
            })
@@ -64,8 +64,12 @@ export class SchemaObject<Schema extends TObject> {
       return this._config;
    }
 
+   clone() {
+      return structuredClone(this._config);
+   }
+
    async set(config: Static<Schema>, noEmit?: boolean): Promise<Static<Schema>> {
-      const valid = parse(this._schema, cloneDeep(config) as any, {
+      const valid = parse(this._schema, structuredClone(config) as any, {
          forceParse: true,
          skipMark: this.isForceParse()
       });
@@ -114,7 +118,7 @@ export class SchemaObject<Schema extends TObject> {
    }
 
    async patch(path: string, value: any): Promise<[Partial<Static<Schema>>, Static<Schema>]> {
-      const current = cloneDeep(this._config);
+      const current = this.clone();
       const partial = path.length > 0 ? (set({}, path, value) as Partial<Static<Schema>>) : value;
 
       this.throwIfRestricted(partial);
@@ -122,15 +126,20 @@ export class SchemaObject<Schema extends TObject> {
 
       // overwrite arrays and primitives, only deep merge objects
       // @ts-ignore
-      const config = mergeWith(current, partial, (objValue, srcValue) => {
+      //console.log("---alt:new", _jsonp(mergeObject(current, partial)));
+      const config = mergeObjectWith(current, partial, (objValue, srcValue) => {
          if (Array.isArray(objValue) && Array.isArray(srcValue)) {
             return srcValue;
          }
       });
+      //console.log("---new", _jsonp(config));
 
       //console.log("overwritePaths", this.options?.overwritePaths);
       if (this.options?.overwritePaths) {
-         const keys = getFullPathKeys(value).map((k) => path + "." + k);
+         const keys = getFullPathKeys(value).map((k) => {
+            // only prepend path if given
+            return path.length > 0 ? path + "." + k : k;
+         });
          const overwritePaths = keys.filter((k) => {
             return this.options?.overwritePaths?.some((p) => {
                if (typeof p === "string") {
@@ -161,14 +170,14 @@ export class SchemaObject<Schema extends TObject> {
          }
       }
 
-      //console.log("patch", { path, value, partial, config, current });
+      //console.log("patch", _jsonp({ path, value, partial, config, current }));
 
       const newConfig = await this.set(config);
       return [partial, newConfig];
    }
 
    async overwrite(path: string, value: any): Promise<[Partial<Static<Schema>>, Static<Schema>]> {
-      const current = cloneDeep(this._config);
+      const current = this.clone();
       const partial = path.length > 0 ? (set({}, path, value) as Partial<Static<Schema>>) : value;
 
       this.throwIfRestricted(partial);
@@ -189,7 +198,7 @@ export class SchemaObject<Schema extends TObject> {
       if (p.length > 1) {
          const parent = p.slice(0, -1).join(".");
          if (!has(this._config, parent)) {
-            console.log("parent", parent, JSON.stringify(this._config, null, 2));
+            //console.log("parent", parent, JSON.stringify(this._config, null, 2));
             throw new Error(`Parent path "${parent}" does not exist`);
          }
       }
@@ -204,7 +213,7 @@ export class SchemaObject<Schema extends TObject> {
          throw new Error(`Path "${path}" does not exist`);
       }
 
-      const current = cloneDeep(this._config);
+      const current = this.clone();
       const removed = get(current, path) as Partial<Static<Schema>>;
       const config = omit(current, path);
       const newConfig = await this.set(config);

app/src/core/object/query/query.ts

@@ -49,7 +49,7 @@ type LiteralExpressionCondition<Exps extends Expressions> = {
    [key: string]: Primitive | ExpressionCondition<Exps>;
 };
 
-const OperandOr = "$or";
+const OperandOr = "$or" as const;
 type OperandCondition<Exps extends Expressions> = {
    [OperandOr]?: LiteralExpressionCondition<Exps> | ExpressionCondition<Exps>;
 };

app/src/core/server/flash.ts

@@ -4,14 +4,12 @@ import { setCookie } from "hono/cookie";
 const flash_key = "__bknd_flash";
 export type FlashMessageType = "error" | "warning" | "success" | "info";
 
-export async function addFlashMessage(
-   c: Context,
-   message: string,
-   type: FlashMessageType = "info"
-) {
-   setCookie(c, flash_key, JSON.stringify({ type, message }), {
-      path: "/"
-   });
+export function addFlashMessage(c: Context, message: string, type: FlashMessageType = "info") {
+   if (c.req.header("Accept")?.includes("text/html")) {
+      setCookie(c, flash_key, JSON.stringify({ type, message }), {
+         path: "/"
+      });
+   }
 }
 
 function getCookieValue(name) {

app/src/core/utils/index.ts

@@ -11,3 +11,5 @@ export * from "./crypto";
 export * from "./uuid";
 export { FromSchema } from "./typebox/from-schema";
 export * from "./test";
+export * from "./runtime";
+export * from "./numbers";

app/src/core/utils/numbers.ts

@@ -0,0 +1,5 @@
+export function clampNumber(value: number, min: number, max: number): number {
+   const lower = Math.min(min, max);
+   const upper = Math.max(min, max);
+   return Math.max(lower, Math.min(value, upper));
+}

app/src/core/utils/objects.ts

@@ -4,6 +4,28 @@ export function _jsonp(obj: any, space = 2): string {
    return JSON.stringify(obj, null, space);
 }
 
+export function isPlainObject(value: unknown): value is Record<string, unknown> {
+   return Object.prototype.toString.call(value) === "[object Object]";
+}
+
+export function isObject(value: unknown): value is Record<string, unknown> {
+   return value !== null && typeof value === "object";
+}
+
+export function omitKeys<T extends object, K extends keyof T>(
+   obj: T,
+   keys_: readonly K[]
+): Omit<T, Extract<K, keyof T>> {
+   const keys = new Set(keys_);
+   const result = {} as Omit<T, Extract<K, keyof T>>;
+   for (const [key, value] of Object.entries(obj) as [keyof T, T[keyof T]][]) {
+      if (!keys.has(key as K)) {
+         (result as any)[key] = value;
+      }
+   }
+   return result;
+}
+
 export function safelyParseObjectValues<T extends { [key: string]: any }>(obj: T): T {
    return Object.entries(obj).reduce((acc, [key, value]) => {
       try {
@@ -97,15 +119,6 @@ export function objectEach<T extends Record<string, any>, U>(
    );
 }
 
-/**
- * Simple object check.
- * @param item
- * @returns {boolean}
- */
-export function isObject(item) {
-   return item && typeof item === "object" && !Array.isArray(item);
-}
-
 /**
  * Deep merge two objects.
  * @param target
@@ -197,3 +210,152 @@ export function objectCleanEmpty<Obj extends { [key: string]: any }>(obj: Obj):
       return acc;
    }, {} as any);
 }
+
+/**
+ * Lodash's merge implementation caused issues in Next.js environments
+ * From: https://thescottyjam.github.io/snap.js/#!/nolodash/merge
+ * NOTE: This mutates `object`. It also may mutate anything that gets attached to `object` during the merge.
+ * @param object
+ * @param sources
+ */
+export function mergeObject(object, ...sources) {
+   for (const source of sources) {
+      for (const [key, value] of Object.entries(source)) {
+         if (value === undefined) {
+            continue;
+         }
+
+         // These checks are a week attempt at mimicking the various edge-case behaviors
+         // that Lodash's `_.merge()` exhibits. Feel free to simplify and
+         // remove checks that you don't need.
+         if (!isPlainObject(value) && !Array.isArray(value)) {
+            object[key] = value;
+         } else if (Array.isArray(value) && !Array.isArray(object[key])) {
+            object[key] = value;
+         } else if (!isObject(object[key])) {
+            object[key] = value;
+         } else {
+            mergeObject(object[key], value);
+         }
+      }
+   }
+
+   return object;
+}
+
+/**
+ * Lodash's mergeWith implementation caused issues in Next.js environments
+ * From: https://thescottyjam.github.io/snap.js/#!/nolodash/mergeWith
+ * NOTE: This mutates `object`. It also may mutate anything that gets attached to `object` during the merge.
+ * @param object
+ * @param sources
+ * @param customizer
+ */
+export function mergeObjectWith(object, source, customizer) {
+   for (const [key, value] of Object.entries(source)) {
+      const mergedValue = customizer(object[key], value, key, object, source);
+      if (mergedValue !== undefined) {
+         object[key] = mergedValue;
+         continue;
+      }
+      // Otherwise, fall back to default behavior
+
+      if (value === undefined) {
+         continue;
+      }
+
+      // These checks are a week attempt at mimicking the various edge-case behaviors
+      // that Lodash's `_.merge()` exhibits. Feel free to simplify and
+      // remove checks that you don't need.
+      if (!isPlainObject(value) && !Array.isArray(value)) {
+         object[key] = value;
+      } else if (Array.isArray(value) && !Array.isArray(object[key])) {
+         object[key] = value;
+      } else if (!isObject(object[key])) {
+         object[key] = value;
+      } else {
+         mergeObjectWith(object[key], value, customizer);
+      }
+   }
+
+   return object;
+}
+
+export function isEqual(value1: any, value2: any): boolean {
+   // Each type corresponds to a particular comparison algorithm
+   const getType = (value: any) => {
+      if (value !== Object(value)) return "primitive";
+      if (Array.isArray(value)) return "array";
+      if (value instanceof Map) return "map";
+      if (value != null && [null, Object.prototype].includes(Object.getPrototypeOf(value)))
+         return "plainObject";
+      if (value instanceof Function) return "function";
+      throw new Error(
+         `deeply comparing an instance of type ${value1.constructor?.name} is not supported.`
+      );
+   };
+
+   const type = getType(value1);
+   if (type !== getType(value2)) {
+      return false;
+   }
+
+   if (type === "primitive") {
+      return value1 === value2 || (Number.isNaN(value1) && Number.isNaN(value2));
+   } else if (type === "array") {
+      return (
+         value1.length === value2.length &&
+         value1.every((iterValue: any, i: number) => isEqual(iterValue, value2[i]))
+      );
+   } else if (type === "map") {
+      // In this particular implementation, map keys are not
+      // being deeply compared, only map values.
+      return (
+         value1.size === value2.size &&
+         [...value1].every(([iterKey, iterValue]) => {
+            return value2.has(iterKey) && isEqual(iterValue, value2.get(iterKey));
+         })
+      );
+   } else if (type === "plainObject") {
+      const value1AsMap = new Map(Object.entries(value1));
+      const value2AsMap = new Map(Object.entries(value2));
+      return (
+         value1AsMap.size === value2AsMap.size &&
+         [...value1AsMap].every(([iterKey, iterValue]) => {
+            return value2AsMap.has(iterKey) && isEqual(iterValue, value2AsMap.get(iterKey));
+         })
+      );
+   } else if (type === "function") {
+      // just check signature
+      return value1.toString() === value2.toString();
+   } else {
+      throw new Error("Unreachable");
+   }
+}
+
+export function getPath(
+   object: object,
+   _path: string | (string | number)[],
+   defaultValue = undefined
+): any {
+   const path = typeof _path === "string" ? _path.split(/[.\[\]\"]+/).filter((x) => x) : _path;
+
+   if (path.length === 0) {
+      return object;
+   }
+
+   try {
+      const [head, ...tail] = path;
+      if (!head || !(head in object)) {
+         return defaultValue;
+      }
+
+      return getPath(object[head], tail, defaultValue);
+   } catch (error) {
+      if (typeof defaultValue !== "undefined") {
+         return defaultValue;
+      }
+
+      throw new Error(`Invalid path: ${path.join(".")}`);
+   }
+}

app/src/core/utils/runtime.ts

@@ -0,0 +1,41 @@
+import { getRuntimeKey as honoGetRuntimeKey } from "hono/adapter";
+
+/**
+ * Adds additional checks for nextjs
+ */
+export function getRuntimeKey(): string {
+   const global = globalThis as any;
+
+   // Detect Next.js server-side runtime
+   if (global?.process?.env?.NEXT_RUNTIME === "nodejs") {
+      return "nextjs";
+   }
+
+   // Detect Next.js edge runtime
+   if (global?.process?.env?.NEXT_RUNTIME === "edge") {
+      return "nextjs-edge";
+   }
+
+   // Detect Next.js client-side runtime
+   // @ts-ignore
+   if (typeof window !== "undefined" && window.__NEXT_DATA__) {
+      return "nextjs-client";
+   }
+
+   return honoGetRuntimeKey();
+}
+
+const features = {
+   // supports the redirect of not full qualified addresses
+   // not supported in nextjs
+   redirects_non_fq: true
+};
+
+export function runtimeSupports(feature: keyof typeof features) {
+   const runtime = getRuntimeKey();
+   if (runtime.startsWith("nextjs")) {
+      features.redirects_non_fq = false;
+   }
+
+   return features[feature];
+}

app/src/core/utils/strings.ts

@@ -81,9 +81,12 @@ export function identifierToHumanReadable(str: string) {
       case "SCREAMING_SNAKE_CASE":
          return snakeToPascalWithSpaces(str.toLowerCase());
       case "unknown":
-         return str;
+         return ucFirst(str);
    }
 }
+export function autoFormatString(str: string) {
+   return identifierToHumanReadable(str);
+}
 
 export function kebabToPascalWithSpaces(str: string): string {
    return str.split("-").map(ucFirst).join(" ");
@@ -104,3 +107,14 @@ export function replaceSimplePlaceholders(str: string, vars: Record<string, any>
       return key in vars ? vars[key] : match;
    });
 }
+
+export function patternMatch(target: string, pattern: RegExp | string): boolean {
+   if (pattern instanceof RegExp) {
+      return pattern.test(target);
+   } else if (typeof pattern === "string" && pattern.startsWith("/")) {
+      return new RegExp(pattern).test(target);
+   } else if (typeof pattern === "string") {
+      return target.startsWith(pattern);
+   }
+   return false;
+}

app/src/core/utils/test.ts

@@ -7,7 +7,7 @@ const _oldConsoles = {
 
 export async function withDisabledConsole<R>(
    fn: () => Promise<R>,
-   severities: ConsoleSeverity[] = ["log"]
+   severities: ConsoleSeverity[] = ["log", "warn", "error"]
 ): Promise<R> {
    const _oldConsoles = {
       log: console.log,
@@ -30,7 +30,7 @@ export async function withDisabledConsole<R>(
    }
 }
 
-export function disableConsoleLog(severities: ConsoleSeverity[] = ["log"]) {
+export function disableConsoleLog(severities: ConsoleSeverity[] = ["log", "warn"]) {
    severities.forEach((severity) => {
       console[severity] = () => null;
    });

app/src/core/utils/typebox/index.ts

@@ -21,7 +21,6 @@ import {
    type ValueErrorIterator
 } from "@sinclair/typebox/errors";
 import { Check, Default, Value, type ValueError } from "@sinclair/typebox/value";
-import { cloneDeep } from "lodash-es";
 
 export type RecursivePartial<T> = {
    [P in keyof T]?: T[P] extends (infer U)[]
@@ -73,7 +72,7 @@ export class TypeInvalidError extends Error {
 }
 
 export function stripMark<O = any>(obj: O) {
-   const newObj = cloneDeep(obj);
+   const newObj = structuredClone(obj);
    mark(newObj, false);
    return newObj as O;
 }
@@ -115,6 +114,7 @@ export function parse<Schema extends TSchema = TSchema>(
    } else if (options?.onError) {
       options.onError(Errors(schema, data));
    } else {
+      //console.warn("errors", JSON.stringify([...Errors(schema, data)], null, 2));
       throw new TypeInvalidError(schema, data);
    }
 

app/src/data/AppData.ts

@@ -13,7 +13,15 @@ import { type AppDataConfig, dataConfigSchema } from "./data-schema";
 
 export class AppData extends Module<typeof dataConfigSchema> {
    override async build() {
-      const entities = transformObject(this.config.entities ?? {}, (entityConfig, name) => {
+      const {
+         entities: _entities = {},
+         relations: _relations = {},
+         indices: _indices = {}
+      } = this.config;
+
+      this.ctx.logger.context("AppData").log("building with entities", Object.keys(_entities));
+
+      const entities = transformObject(_entities, (entityConfig, name) => {
          return constructEntity(name, entityConfig);
       });
 
@@ -21,14 +29,14 @@ export class AppData extends Module<typeof dataConfigSchema> {
          const name = typeof _e === "string" ? _e : _e.name;
          const entity = entities[name];
          if (entity) return entity;
-         throw new Error(`Entity "${name}" not found`);
+         throw new Error(`[AppData] Entity "${name}" not found`);
       };
 
-      const relations = transformObject(this.config.relations ?? {}, (relation) =>
+      const relations = transformObject(_relations, (relation) =>
          constructRelation(relation, _entity)
       );
 
-      const indices = transformObject(this.config.indices ?? {}, (index, name) => {
+      const indices = transformObject(_indices, (index, name) => {
          const entity = _entity(index.entity)!;
          const fields = index.fields.map((f) => entity.field(f)!);
          return new EntityIndex(entity, fields, index.unique, name);
@@ -52,6 +60,7 @@ export class AppData extends Module<typeof dataConfigSchema> {
       );
       this.ctx.guard.registerPermissions(Object.values(DataPermissions));
 
+      this.ctx.logger.clear();
       this.setBuilt();
    }
 
@@ -69,18 +78,9 @@ export class AppData extends Module<typeof dataConfigSchema> {
    }
 
    override getOverwritePaths() {
-      return [
-         /^entities\..*\.config$/,
-         /^entities\..*\.fields\..*\.config$/
-         ///^entities\..*\.fields\..*\.config\.schema$/
-      ];
+      return [/^entities\..*\.config$/, /^entities\..*\.fields\..*\.config$/];
    }
 
-   /*registerController(server: AppServer) {
-      console.log("adding data controller to", this.basepath);
-      server.add(this.basepath, new DataController(this.em));
-   }*/
-
    override toJSON(secrets?: boolean): AppDataConfig {
       return {
          ...this.config,

app/src/data/api/DataApi.ts

@@ -1,15 +1,17 @@
 import type { DB } from "core";
-import type { EntityData, RepoQuery, RepositoryResponse } from "data";
+import type { EntityData, RepoQuery, RepoQueryIn, RepositoryResponse } from "data";
 import { type BaseModuleApiOptions, ModuleApi, type PrimaryFieldType } from "modules";
 
 export type DataApiOptions = BaseModuleApiOptions & {
-   defaultQuery?: Partial<RepoQuery>;
+   queryLengthLimit: number;
+   defaultQuery: Partial<RepoQuery>;
 };
 
 export class DataApi extends ModuleApi<DataApiOptions> {
    protected override getDefaultOptions(): Partial<DataApiOptions> {
       return {
          basepath: "/api/data",
+         queryLengthLimit: 1000,
          defaultQuery: {
             limit: 10
          }
@@ -19,26 +21,32 @@ export class DataApi extends ModuleApi<DataApiOptions> {
    readOne<E extends keyof DB | string, Data = E extends keyof DB ? DB[E] : EntityData>(
       entity: E,
       id: PrimaryFieldType,
-      query: Partial<Omit<RepoQuery, "where" | "limit" | "offset">> = {}
+      query: Omit<RepoQueryIn, "where" | "limit" | "offset"> = {}
    ) {
       return this.get<Pick<RepositoryResponse<Data>, "meta" | "data">>([entity as any, id], query);
    }
 
    readMany<E extends keyof DB | string, Data = E extends keyof DB ? DB[E] : EntityData>(
       entity: E,
-      query: Partial<RepoQuery> = {}
+      query: RepoQueryIn = {}
    ) {
-      return this.get<Pick<RepositoryResponse<Data[]>, "meta" | "data">>(
-         [entity as any],
-         query ?? this.options.defaultQuery
-      );
+      type T = Pick<RepositoryResponse<Data[]>, "meta" | "data">;
+
+      const input = query ?? this.options.defaultQuery;
+      const req = this.get<T>([entity as any], input);
+
+      if (req.request.url.length <= this.options.queryLengthLimit) {
+         return req;
+      }
+
+      return this.post<T>([entity as any, "query"], input);
    }
 
    readManyByReference<
       E extends keyof DB | string,
       R extends keyof DB | string,
       Data = R extends keyof DB ? DB[R] : EntityData
-   >(entity: E, id: PrimaryFieldType, reference: R, query: Partial<RepoQuery> = {}) {
+   >(entity: E, id: PrimaryFieldType, reference: R, query: RepoQueryIn = {}) {
       return this.get<Pick<RepositoryResponse<Data[]>, "meta" | "data">>(
          [entity as any, id, reference],
          query ?? this.options.defaultQuery

app/src/data/api/DataController.ts

@@ -1,32 +1,26 @@
-import { type ClassController, isDebug, tbValidator as tb } from "core";
-import { StringEnum, Type, objectCleanEmpty, objectTransform } from "core/utils";
+import { isDebug, tbValidator as tb } from "core";
+import { StringEnum, Type } from "core/utils";
 import {
    DataPermissions,
    type EntityData,
    type EntityManager,
-   FieldClassMap,
    type MutatorResponse,
-   PrimaryField,
    type RepoQuery,
    type RepositoryResponse,
-   TextField,
    querySchema
 } from "data";
-import { Hono } from "hono";
 import type { Handler } from "hono/types";
 import type { ModuleBuildContext } from "modules";
+import { Controller } from "modules/Controller";
 import * as SystemPermissions from "modules/permissions";
-import { type AppDataConfig, FIELDS } from "../data-schema";
+import type { AppDataConfig } from "../data-schema";
 
-export class DataController implements ClassController {
+export class DataController extends Controller {
    constructor(
       private readonly ctx: ModuleBuildContext,
       private readonly config: AppDataConfig
    ) {
-      /*console.log(
-         "data controller",
-         this.em.entities.map((e) => e.name)
-      );*/
+      super();
    }
 
    get em(): EntityManager<any> {
@@ -74,8 +68,10 @@ export class DataController implements ClassController {
       }
    }
 
-   getController(): Hono<any> {
-      const hono = new Hono();
+   override getController() {
+      const { permission, auth } = this.middlewares;
+      const hono = this.create().use(auth(), permission(SystemPermissions.accessApi));
+
       const definedEntities = this.em.entities.map((e) => e.name);
       const tbNumber = Type.Transform(Type.String({ pattern: "^[1-9][0-9]{0,}$" }))
          .Decode(Number.parseInt)
@@ -89,11 +85,6 @@ export class DataController implements ClassController {
          return func;
       }
 
-      hono.use("*", async (c, next) => {
-         this.ctx.guard.throwUnlessGranted(SystemPermissions.accessApi);
-         await next();
-      });
-
       // info
       hono.get(
          "/",
@@ -104,9 +95,7 @@ export class DataController implements ClassController {
       );
 
       // sync endpoint
-      hono.get("/sync", async (c) => {
-         this.guard.throwUnlessGranted(DataPermissions.databaseSync);
-
+      hono.get("/sync", permission(DataPermissions.databaseSync), async (c) => {
          const force = c.req.query("force") === "1";
          const drop = c.req.query("drop") === "1";
          //console.log("force", force);
@@ -126,10 +115,9 @@ export class DataController implements ClassController {
          // fn: count
          .post(
             "/:entity/fn/count",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity } = c.req.valid("param");
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -143,10 +131,9 @@ export class DataController implements ClassController {
          // fn: exists
          .post(
             "/:entity/fn/exists",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity } = c.req.valid("param");
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -163,8 +150,7 @@ export class DataController implements ClassController {
        */
       hono
          // read entity schema
-         .get("/schema.json", async (c) => {
-            this.guard.throwUnlessGranted(DataPermissions.entityRead);
+         .get("/schema.json", permission(DataPermissions.entityRead), async (c) => {
             const $id = `${this.config.basepath}/schema.json`;
             const schemas = Object.fromEntries(
                this.em.entities.map((e) => [
@@ -183,6 +169,7 @@ export class DataController implements ClassController {
          // read schema
          .get(
             "/schemas/:entity/:context?",
+            permission(DataPermissions.entityRead),
             tb(
                "param",
                Type.Object({
@@ -191,8 +178,6 @@ export class DataController implements ClassController {
                })
             ),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                //console.log("request", c.req.raw);
                const { entity, context } = c.req.param();
                if (!this.entityExists(entity)) {
@@ -216,11 +201,10 @@ export class DataController implements ClassController {
          // read many
          .get(
             "/:entity",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             tb("query", querySchema),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                //console.log("request", c.req.raw);
                const { entity } = c.req.param();
                if (!this.entityExists(entity)) {
@@ -238,6 +222,7 @@ export class DataController implements ClassController {
          // read one
          .get(
             "/:entity/:id",
+            permission(DataPermissions.entityRead),
             tb(
                "param",
                Type.Object({
@@ -246,11 +231,7 @@ export class DataController implements ClassController {
                })
             ),
             tb("query", querySchema),
-            /*zValidator("param", z.object({ entity: z.string(), id: z.coerce.number() })),
-            zValidator("query", repoQuerySchema),*/
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity, id } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -264,6 +245,7 @@ export class DataController implements ClassController {
          // read many by reference
          .get(
             "/:entity/:id/:reference",
+            permission(DataPermissions.entityRead),
             tb(
                "param",
                Type.Object({
@@ -274,8 +256,6 @@ export class DataController implements ClassController {
             ),
             tb("query", querySchema),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity, id, reference } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -292,17 +272,16 @@ export class DataController implements ClassController {
          // func query
          .post(
             "/:entity/query",
+            permission(DataPermissions.entityRead),
             tb("param", Type.Object({ entity: Type.String() })),
             tb("json", querySchema),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityRead);
-
                const { entity } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
                }
                const options = (await c.req.valid("json")) as RepoQuery;
-               console.log("options", options);
+               //console.log("options", options);
                const result = await this.em.repository(entity).findMany(options);
 
                return c.json(this.repoResult(result), { status: result.data ? 200 : 404 });
@@ -314,25 +293,27 @@ export class DataController implements ClassController {
        */
       // insert one
       hono
-         .post("/:entity", tb("param", Type.Object({ entity: Type.String() })), async (c) => {
-            this.guard.throwUnlessGranted(DataPermissions.entityCreate);
+         .post(
+            "/:entity",
+            permission(DataPermissions.entityCreate),
+            tb("param", Type.Object({ entity: Type.String() })),
+            async (c) => {
+               const { entity } = c.req.param();
+               if (!this.entityExists(entity)) {
+                  return c.notFound();
+               }
+               const body = (await c.req.json()) as EntityData;
+               const result = await this.em.mutator(entity).insertOne(body);
 
-            const { entity } = c.req.param();
-            if (!this.entityExists(entity)) {
-               return c.notFound();
+               return c.json(this.mutatorResult(result), 201);
             }
-            const body = (await c.req.json()) as EntityData;
-            const result = await this.em.mutator(entity).insertOne(body);
-
-            return c.json(this.mutatorResult(result), 201);
-         })
+         )
          // update one
          .patch(
             "/:entity/:id",
+            permission(DataPermissions.entityUpdate),
             tb("param", Type.Object({ entity: Type.String(), id: tbNumber })),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityUpdate);
-
                const { entity, id } = c.req.param();
                if (!this.entityExists(entity)) {
                   return c.notFound();
@@ -346,6 +327,8 @@ export class DataController implements ClassController {
          // delete one
          .delete(
             "/:entity/:id",
+
+            permission(DataPermissions.entityDelete),
             tb("param", Type.Object({ entity: Type.String(), id: tbNumber })),
             async (c) => {
                this.guard.throwUnlessGranted(DataPermissions.entityDelete);
@@ -363,11 +346,10 @@ export class DataController implements ClassController {
          // delete many
          .delete(
             "/:entity",
+            permission(DataPermissions.entityDelete),
             tb("param", Type.Object({ entity: Type.String() })),
             tb("json", querySchema.properties.where),
             async (c) => {
-               this.guard.throwUnlessGranted(DataPermissions.entityDelete);
-
                //console.log("request", c.req.raw);
                const { entity } = c.req.param();
                if (!this.entityExists(entity)) {

app/src/data/connection/SqliteLocalConnection.ts

@@ -1,6 +1,5 @@
-import type { DatabaseIntrospector, SqliteDatabase } from "kysely";
+import { type DatabaseIntrospector, ParseJSONResultsPlugin, type SqliteDatabase } from "kysely";
 import { Kysely, SqliteDialect } from "kysely";
-import { DeserializeJsonValuesPlugin } from "../plugins/DeserializeJsonValuesPlugin";
 import { SqliteConnection } from "./SqliteConnection";
 import { SqliteIntrospector } from "./SqliteIntrospector";
 
@@ -14,7 +13,7 @@ class CustomSqliteDialect extends SqliteDialect {
 
 export class SqliteLocalConnection extends SqliteConnection {
    constructor(private database: SqliteDatabase) {
-      const plugins = [new DeserializeJsonValuesPlugin()];
+      const plugins = [new ParseJSONResultsPlugin()];
       const kysely = new Kysely({
          dialect: new CustomSqliteDialect({ database }),
          plugins

app/src/data/entities/Entity.ts

@@ -98,8 +98,8 @@ export class Entity<
 
    getDefaultSort() {
       return {
-         by: this.config.sort_field,
-         dir: this.config.sort_dir
+         by: this.config.sort_field ?? "id",
+         dir: this.config.sort_dir ?? "asc"
       };
    }
 
@@ -140,7 +140,7 @@ export class Entity<
       return this.fields.find((field) => field.name === name);
    }
 
-   __experimental_replaceField(name: string, field: Field) {
+   __replaceField(name: string, field: Field) {
       const index = this.fields.findIndex((f) => f.name === name);
       if (index === -1) {
          throw new Error(`Field "${name}" not found on entity "${this.name}"`);
@@ -192,14 +192,41 @@ export class Entity<
       this.data = data;
    }
 
-   isValidData(data: EntityData, context: TActionContext, explain?: boolean): boolean {
+   // @todo: add tests
+   isValidData(
+      data: EntityData,
+      context: TActionContext,
+      options?: {
+         explain?: boolean;
+         ignoreUnknown?: boolean;
+      }
+   ): boolean {
+      if (typeof data !== "object") {
+         if (options?.explain) {
+            throw new Error(`Entity "${this.name}" data must be an object`);
+         }
+      }
+
       const fields = this.getFillableFields(context, false);
-      //const fields = this.fields;
-      //console.log("data", data);
+
+      if (options?.ignoreUnknown !== true) {
+         const field_names = fields.map((f) => f.name);
+         const given_keys = Object.keys(data);
+         const unknown_keys = given_keys.filter((key) => !field_names.includes(key));
+
+         if (unknown_keys.length > 0) {
+            if (options?.explain) {
+               throw new Error(
+                  `Entity "${this.name}" data must only contain known keys, unknown: "${unknown_keys}"`
+               );
+            }
+         }
+      }
+
       for (const field of fields) {
          if (!field.isValid(data[field.name], context)) {
             console.log("Entity.isValidData:invalid", context, field.name, data[field.name]);
-            if (explain) {
+            if (options?.explain) {
                throw new Error(`Field "${field.name}" has invalid data: "${data[field.name]}"`);
             }
 

app/src/data/entities/EntityManager.ts

@@ -99,17 +99,30 @@ export class EntityManager<TBD extends object = DefaultDB> {
       this.entities.push(entity);
    }
 
-   entity(e: Entity | keyof TBD | string): Entity {
-      let entity: Entity | undefined;
-      if (typeof e === "string") {
-         entity = this.entities.find((entity) => entity.name === e);
-      } else if (e instanceof Entity) {
-         entity = e;
+   __replaceEntity(entity: Entity, name: string | undefined = entity.name) {
+      const entityIndex = this._entities.findIndex((e) => e.name === name);
+
+      if (entityIndex === -1) {
+         throw new Error(`Entity "${name}" not found and cannot be replaced`);
       }
 
+      this._entities[entityIndex] = entity;
+
+      // caused issues because this.entity() was using a reference (for when initial config was given)
+   }
+
+   entity<Silent extends true | false = false>(
+      e: Entity | keyof TBD | string,
+      silent?: Silent
+   ): Silent extends true ? Entity | undefined : Entity {
+      // make sure to always retrieve by name
+      const entity = this.entities.find((entity) =>
+         e instanceof Entity ? entity.name === e.name : entity.name === e
+      );
+
       if (!entity) {
-         // @ts-ignore
-         throw new EntityNotDefinedException(e instanceof Entity ? e.name : e);
+         if (silent === true) return undefined as any;
+         throw new EntityNotDefinedException(e instanceof Entity ? e.name : (e as string));
       }
 
       return entity;

app/src/data/entities/Mutator.ts

@@ -132,14 +132,17 @@ export class Mutator<
          throw new Error(`Creation of system entity "${entity.name}" is disabled`);
       }
 
-      // @todo: establish the original order from "data"
+      const result = await this.emgr.emit(
+         new Mutator.Events.MutatorInsertBefore({ entity, data: data as any })
+      );
+
+      // if listener returned, take what's returned
+      const _data = result.returned ? result.params.data : data;
       const validatedData = {
          ...entity.getDefaultObject(),
-         ...(await this.getValidatedData(data, "create"))
+         ...(await this.getValidatedData(_data, "create"))
       };
 
-      await this.emgr.emit(new Mutator.Events.MutatorInsertBefore({ entity, data: validatedData }));
-
       // check if required fields are present
       const required = entity.getRequiredFields();
       for (const field of required) {
@@ -169,16 +172,17 @@ export class Mutator<
          throw new Error("ID must be provided for update");
       }
 
-      const validatedData = await this.getValidatedData(data, "update");
-
-      await this.emgr.emit(
+      const result = await this.emgr.emit(
          new Mutator.Events.MutatorUpdateBefore({
             entity,
             entityId: id,
-            data: validatedData as any
+            data
          })
       );
 
+      const _data = result.returned ? result.params.data : data;
+      const validatedData = await this.getValidatedData(_data, "update");
+
       const query = this.conn
          .updateTable(entity.name)
          .set(validatedData as any)

app/src/data/entities/query/Repository.ts

@@ -58,14 +58,14 @@ export class Repository<TBD extends object = DefaultDB, TB extends keyof TBD = a
    }
 
    private cloneFor(entity: Entity) {
-      return new Repository(this.em, entity, this.emgr);
+      return new Repository(this.em, this.em.entity(entity), this.emgr);
    }
 
    private get conn() {
       return this.em.connection.kysely;
    }
 
-   private getValidOptions(options?: Partial<RepoQuery>): RepoQuery {
+   getValidOptions(options?: Partial<RepoQuery>): RepoQuery {
       const entity = this.entity;
       // @todo: if not cloned deep, it will keep references and error if multiple requests come in
       const validated = {
@@ -94,23 +94,19 @@ export class Repository<TBD extends object = DefaultDB, TB extends keyof TBD = a
          if (invalid.length > 0) {
             throw new InvalidSearchParamsException(
                `Invalid select field(s): ${invalid.join(", ")}`
-            );
+            ).context({
+               entity: entity.name,
+               valid: validated.select
+            });
          }
 
          validated.select = options.select;
       }
 
-      if (options.with && options.with.length > 0) {
-         for (const entry of options.with) {
-            const related = this.em.relationOf(entity.name, entry);
-            if (!related) {
-               throw new InvalidSearchParamsException(
-                  `WITH: "${entry}" is not a relation of "${entity.name}"`
-               );
-            }
-
-            validated.with.push(entry);
-         }
+      if (options.with) {
+         const depth = WithBuilder.validateWiths(this.em, entity.name, options.with);
+         // @todo: determine allowed depth
+         validated.with = options.with;
       }
 
       if (options.join && options.join.length > 0) {
@@ -232,43 +228,79 @@ export class Repository<TBD extends object = DefaultDB, TB extends keyof TBD = a
       return { ...response, data: data[0]! };
    }
 
-   private buildQuery(
+   addOptionsToQueryBuilder(
+      _qb?: RepositoryQB,
       _options?: Partial<RepoQuery>,
-      exclude_options: (keyof RepoQuery)[] = []
-   ): { qb: RepositoryQB; options: RepoQuery } {
+      config?: {
+         validate?: boolean;
+         ignore?: (keyof RepoQuery)[];
+         alias?: string;
+         defaults?: Pick<RepoQuery, "limit" | "offset">;
+      }
+   ) {
       const entity = this.entity;
-      const options = this.getValidOptions(_options);
+      let qb = _qb ?? (this.conn.selectFrom(entity.name) as RepositoryQB);
 
-      const alias = entity.name;
+      const options = config?.validate !== false ? this.getValidOptions(_options) : _options;
+      if (!options) return qb;
+
+      const alias = config?.alias ?? entity.name;
       const aliased = (field: string) => `${alias}.${field}`;
-      let qb = this.conn
-         .selectFrom(entity.name)
-         .select(entity.getAliasedSelectFrom(options.select, alias));
+      const ignore = config?.ignore ?? [];
+      const defaults = {
+         limit: 10,
+         offset: 0,
+         ...config?.defaults
+      };
 
-      //console.log("build query options", options);
-      if (!exclude_options.includes("with") && options.with) {
+      /*console.log("build query options", {
+         entity: entity.name,
+         options,
+         config
+      });*/
+
+      if (!ignore.includes("select") && options.select) {
+         qb = qb.select(entity.getAliasedSelectFrom(options.select, alias));
+      }
+
+      if (!ignore.includes("with") && options.with) {
          qb = WithBuilder.addClause(this.em, qb, entity, options.with);
       }
 
-      if (!exclude_options.includes("join") && options.join) {
+      if (!ignore.includes("join") && options.join) {
          qb = JoinBuilder.addClause(this.em, qb, entity, options.join);
       }
 
       // add where if present
-      if (!exclude_options.includes("where") && options.where) {
+      if (!ignore.includes("where") && options.where) {
          qb = WhereBuilder.addClause(qb, options.where);
       }
 
-      if (!exclude_options.includes("limit")) qb = qb.limit(options.limit);
-      if (!exclude_options.includes("offset")) qb = qb.offset(options.offset);
+      if (!ignore.includes("limit")) qb = qb.limit(options.limit ?? defaults.limit);
+      if (!ignore.includes("offset")) qb = qb.offset(options.offset ?? defaults.offset);
 
       // sorting
-      if (!exclude_options.includes("sort")) {
-         qb = qb.orderBy(aliased(options.sort.by), options.sort.dir);
+      if (!ignore.includes("sort")) {
+         qb = qb.orderBy(aliased(options.sort?.by ?? "id"), options.sort?.dir ?? "asc");
       }
 
-      //console.log("options", { _options, options, exclude_options });
-      return { qb, options };
+      return qb as RepositoryQB;
+   }
+
+   private buildQuery(
+      _options?: Partial<RepoQuery>,
+      ignore: (keyof RepoQuery)[] = []
+   ): { qb: RepositoryQB; options: RepoQuery } {
+      const entity = this.entity;
+      const options = this.getValidOptions(_options);
+
+      return {
+         qb: this.addOptionsToQueryBuilder(undefined, options, {
+            ignore,
+            alias: entity.name
+         }),
+         options
+      };
    }
 
    async findId(

app/src/data/entities/query/WhereBuilder.ts

@@ -30,7 +30,7 @@ function key(e: unknown): string {
    return e as string;
 }
 
-const expressions: TExpression<any, any, any>[] = [
+const expressions = [
    exp(
       "$eq",
       (v: Primitive) => isPrimitive(v),

app/src/data/entities/query/WithBuilder.ts

@@ -1,42 +1,82 @@
+import { isObject } from "core/utils";
+import type { KyselyJsonFrom, RepoQuery } from "data";
+import { InvalidSearchParamsException } from "data/errors";
 import type { Entity, EntityManager, RepositoryQB } from "../../entities";
 
 export class WithBuilder {
-   private static buildClause(
+   static addClause(
       em: EntityManager<any>,
       qb: RepositoryQB,
       entity: Entity,
-      withString: string
+      withs: RepoQuery["with"]
    ) {
-      const relation = em.relationOf(entity.name, withString);
-      if (!relation) {
-         throw new Error(`Relation "${withString}" not found`);
+      if (!withs || !isObject(withs)) {
+         console.warn(`'withs' undefined or invalid, given: ${JSON.stringify(withs)}`);
+         return qb;
       }
 
-      const cardinality = relation.ref(withString).cardinality;
-      //console.log("with--builder", { entity: entity.name, withString, cardinality });
-
       const fns = em.connection.fn;
-      const jsonFrom = cardinality === 1 ? fns.jsonObjectFrom : fns.jsonArrayFrom;
-
-      if (!jsonFrom) {
-         throw new Error("Connection does not support jsonObjectFrom/jsonArrayFrom");
-      }
-
-      try {
-         return relation.buildWith(entity, qb, jsonFrom, withString);
-      } catch (e) {
-         throw new Error(`Could not build "with" relation "${withString}": ${(e as any).message}`);
-      }
-   }
-
-   static addClause(em: EntityManager<any>, qb: RepositoryQB, entity: Entity, withs: string[]) {
-      if (withs.length === 0) return qb;
-
       let newQb = qb;
-      for (const entry of withs) {
-         newQb = WithBuilder.buildClause(em, newQb, entity, entry);
+
+      for (const [ref, query] of Object.entries(withs)) {
+         const relation = em.relationOf(entity.name, ref);
+         if (!relation) {
+            throw new Error(`Relation "${entity.name}<>${ref}" not found`);
+         }
+         const cardinality = relation.ref(ref).cardinality;
+         const jsonFrom: KyselyJsonFrom =
+            cardinality === 1 ? fns.jsonObjectFrom : fns.jsonArrayFrom;
+         if (!jsonFrom) {
+            throw new Error("Connection does not support jsonObjectFrom/jsonArrayFrom");
+         }
+
+         const other = relation.other(entity);
+         newQb = newQb.select((eb) => {
+            let subQuery = relation.buildWith(entity, ref)(eb);
+            if (query) {
+               subQuery = em.repo(other.entity).addOptionsToQueryBuilder(subQuery, query as any, {
+                  ignore: ["with", "join", cardinality === 1 ? "limit" : undefined].filter(
+                     Boolean
+                  ) as any
+               });
+            }
+
+            if (query.with) {
+               subQuery = WithBuilder.addClause(em, subQuery, other.entity, query.with as any);
+            }
+
+            return jsonFrom(subQuery).as(other.reference);
+         });
       }
 
       return newQb;
    }
+
+   static validateWiths(em: EntityManager<any>, entity: string, withs: RepoQuery["with"]) {
+      let depth = 0;
+      if (!withs || !isObject(withs)) {
+         withs && console.warn(`'withs' invalid, given: ${JSON.stringify(withs)}`);
+         return depth;
+      }
+
+      const child_depths: number[] = [];
+      for (const [ref, query] of Object.entries(withs)) {
+         const related = em.relationOf(entity, ref);
+         if (!related) {
+            throw new InvalidSearchParamsException(
+               `WITH: "${ref}" is not a relation of "${entity}"`
+            );
+         }
+         depth++;
+
+         if ("with" in query) {
+            child_depths.push(WithBuilder.validateWiths(em, ref, query.with as any));
+         }
+      }
+      if (child_depths.length > 0) {
+         depth += Math.max(...child_depths);
+      }
+
+      return depth;
+   }
 }

app/src/data/events/index.ts

@@ -1,20 +1,48 @@
 import type { PrimaryFieldType } from "core";
-import { Event } from "core/events";
+import { Event, InvalidEventReturn } from "core/events";
 import type { Entity, EntityData } from "../entities";
 import type { RepoQuery } from "../server/data-query-impl";
 
-export class MutatorInsertBefore extends Event<{ entity: Entity; data: EntityData }> {
+export class MutatorInsertBefore extends Event<{ entity: Entity; data: EntityData }, EntityData> {
    static override slug = "mutator-insert-before";
+
+   override validate(data: EntityData) {
+      const { entity } = this.params;
+      if (!entity.isValidData(data, "create")) {
+         throw new InvalidEventReturn("EntityData", "invalid");
+      }
+
+      return this.clone({
+         entity,
+         data
+      });
+   }
 }
 export class MutatorInsertAfter extends Event<{ entity: Entity; data: EntityData }> {
    static override slug = "mutator-insert-after";
 }
-export class MutatorUpdateBefore extends Event<{
-   entity: Entity;
-   entityId: PrimaryFieldType;
-   data: EntityData;
-}> {
+export class MutatorUpdateBefore extends Event<
+   {
+      entity: Entity;
+      entityId: PrimaryFieldType;
+      data: EntityData;
+   },
+   EntityData
+> {
    static override slug = "mutator-update-before";
+
+   override validate(data: EntityData) {
+      const { entity, ...rest } = this.params;
+      if (!entity.isValidData(data, "update")) {
+         throw new InvalidEventReturn("EntityData", "invalid");
+      }
+
+      return this.clone({
+         ...rest,
+         entity,
+         data
+      });
+   }
 }
 export class MutatorUpdateAfter extends Event<{
    entity: Entity;

app/src/data/fields/Field.ts

@@ -12,6 +12,9 @@ import type { HTMLInputTypeAttribute, InputHTMLAttributes } from "react";
 import type { EntityManager } from "../entities";
 import { InvalidFieldConfigException, TransformPersistFailedException } from "../errors";
 
+// @todo: contexts need to be reworked
+// e.g. "table" is irrelevant, because if read is not given, it fails
+
 export const ActionContext = ["create", "read", "update", "delete"] as const;
 export type TActionContext = (typeof ActionContext)[number];
 
@@ -157,8 +160,12 @@ export abstract class Field<
       return this.config.virtual ?? false;
    }
 
-   getLabel(): string {
-      return this.config.label ?? snakeToPascalWithSpaces(this.name);
+   getLabel(options?: { fallback?: boolean }): string | undefined {
+      return this.config.label
+         ? this.config.label
+         : options?.fallback !== false
+           ? snakeToPascalWithSpaces(this.name)
+           : undefined;
    }
 
    getDescription(): string | undefined {

app/src/data/helper.ts

@@ -1,4 +1,4 @@
-import type { EntityData, Field } from "data";
+import type { EntityData, EntityManager, Field } from "data";
 import { transform } from "lodash-es";
 
 export function getDefaultValues(fields: Field[], data: EntityData): EntityData {
@@ -48,3 +48,23 @@ export function getChangeSet(
       {} as typeof formData
    );
 }
+
+export function readableEmJson(_em: EntityManager) {
+   return {
+      entities: _em.entities.map((e) => ({
+         name: e.name,
+         fields: e.fields.map((f) => f.name),
+         type: e.type
+      })),
+      indices: _em.indices.map((i) => ({
+         name: i.name,
+         entity: i.entity.name,
+         fields: i.fields.map((f) => f.name),
+         unique: i.unique
+      })),
+      relations: _em.relations.all.map((r) => ({
+         name: r.getName(),
+         ...r.toJSON()
+      }))
+   };
+}

app/src/data/index.ts

@@ -8,6 +8,7 @@ export * from "./prototype";
 
 export {
    type RepoQuery,
+   type RepoQueryIn,
    defaultQuerySchema,
    querySchema,
    whereSchema
@@ -27,3 +28,5 @@ export const DatabaseEvents = {
 export { MutatorEvents, RepositoryEvents };
 
 export * as DataPermissions from "./permissions";
+
+export { MediaField, type MediaFieldConfig, type MediaItem } from "media/MediaField";

app/src/data/prototype/index.ts

@@ -272,18 +272,22 @@ class EntityManagerPrototype<Entities extends Record<string, Entity>> extends En
    }
 }
 
-type Chained<Fn extends (...args: any[]) => any, Rt = ReturnType<Fn>> = <E extends Entity>(
-   e: E
-) => {
-   [K in keyof Rt]: Rt[K] extends (...args: any[]) => any
-      ? (...args: Parameters<Rt[K]>) => Rt
+type Chained<R extends Record<string, (...args: any[]) => any>> = {
+   [K in keyof R]: R[K] extends (...args: any[]) => any
+      ? (...args: Parameters<R[K]>) => Chained<R>
       : never;
 };
+type ChainedFn<
+   Fn extends (...args: any[]) => Record<string, (...args: any[]) => any>,
+   Return extends ReturnType<Fn> = ReturnType<Fn>
+> = (e: Entity) => {
+   [K in keyof Return]: (...args: Parameters<Return[K]>) => Chained<Return>;
+};
 
 export function em<Entities extends Record<string, Entity>>(
    entities: Entities,
    schema?: (
-      fns: { relation: Chained<typeof relation>; index: Chained<typeof index> },
+      fns: { relation: ChainedFn<typeof relation>; index: ChainedFn<typeof index> },
       entities: Entities
    ) => void
 ) {

app/src/data/relations/EntityRelation.ts

@@ -1,5 +1,5 @@
 import { type Static, Type, parse } from "core/utils";
-import type { SelectQueryBuilder } from "kysely";
+import type { ExpressionBuilder, SelectQueryBuilder } from "kysely";
 import type { Entity, EntityData, EntityManager } from "../entities";
 import {
    type EntityRelationAnchor,
@@ -67,10 +67,8 @@ export abstract class EntityRelation<
     */
    abstract buildWith(
       entity: Entity,
-      qb: KyselyQueryBuilder,
-      jsonFrom: KyselyJsonFrom,
       reference: string
-   ): KyselyQueryBuilder;
+   ): (eb: ExpressionBuilder<any, any>) => KyselyQueryBuilder;
 
    abstract buildJoin(
       entity: Entity,

app/src/data/relations/ManyToManyRelation.ts

@@ -1,4 +1,5 @@
 import { type Static, Type } from "core/utils";
+import type { ExpressionBuilder } from "kysely";
 import { Entity, type EntityManager } from "../entities";
 import { type Field, PrimaryField, VirtualField } from "../fields";
 import type { RepoQuery } from "../server/data-query-impl";
@@ -123,7 +124,7 @@ export class ManyToManyRelation extends EntityRelation<typeof ManyToManyRelation
          .groupBy(groupBy);
    }
 
-   buildWith(entity: Entity, qb: KyselyQueryBuilder, jsonFrom: KyselyJsonFrom) {
+   buildWith(entity: Entity) {
       if (!this.em) {
          throw new Error("EntityManager not set, can't build");
       }
@@ -138,7 +139,29 @@ export class ManyToManyRelation extends EntityRelation<typeof ManyToManyRelation
          (f) => !(f instanceof RelationField || f instanceof PrimaryField)
       );
 
-      return qb.select((eb) => {
+      return (eb: ExpressionBuilder<any, any>) =>
+         eb
+            .selectFrom(other.entity.name)
+            .select((eb2) => {
+               const select: any[] = other.entity.getSelect(other.entity.name);
+               if (additionalFields.length > 0) {
+                  const conn = this.connectionEntity.name;
+                  select.push(
+                     jsonBuildObject(
+                        Object.fromEntries(
+                           additionalFields.map((f) => [f.name, eb2.ref(`${conn}.${f.name}`)])
+                        )
+                     ).as(this.connectionTableMappedName)
+                  );
+               }
+
+               return select;
+            })
+            .whereRef(entityRef, "=", otherRef)
+            .innerJoin(...join)
+            .limit(limit);
+
+      /*return qb.select((eb) => {
          const select: any[] = other.entity.getSelect(other.entity.name);
          // @todo: also add to find by references
          if (additionalFields.length > 0) {
@@ -160,7 +183,7 @@ export class ManyToManyRelation extends EntityRelation<typeof ManyToManyRelation
                .innerJoin(...join)
                .limit(limit)
          ).as(other.reference);
-      });
+      });*/
    }
 
    initialize(em: EntityManager<any>) {

app/src/data/relations/ManyToOneRelation.ts

@@ -1,6 +1,7 @@
 import type { PrimaryFieldType } from "core";
 import { snakeToPascalWithSpaces } from "core/utils";
 import { type Static, Type } from "core/utils";
+import type { ExpressionBuilder } from "kysely";
 import type { Entity, EntityManager } from "../entities";
 import type { RepoQuery } from "../server/data-query-impl";
 import { EntityRelation, type KyselyJsonFrom, type KyselyQueryBuilder } from "./EntityRelation";
@@ -155,23 +156,14 @@ export class ManyToOneRelation extends EntityRelation<typeof ManyToOneRelation.s
       return qb.innerJoin(self.entity.name, entityRef, otherRef).groupBy(groupBy);
    }
 
-   buildWith(entity: Entity, qb: KyselyQueryBuilder, jsonFrom: KyselyJsonFrom, reference: string) {
+   buildWith(entity: Entity, reference: string) {
       const { self, entityRef, otherRef, relationRef } = this.queryInfo(entity, reference);
-      const limit =
-         self.cardinality === 1
-            ? 1
-            : this.config.with_limit ?? ManyToOneRelation.DEFAULTS.with_limit;
-      //console.log("buildWith", entity.name, reference, { limit });
 
-      return qb.select((eb) =>
-         jsonFrom(
-            eb
-               .selectFrom(`${self.entity.name} as ${relationRef}`)
-               .select(self.entity.getSelect(relationRef))
-               .whereRef(entityRef, "=", otherRef)
-               .limit(limit)
-         ).as(relationRef)
-      );
+      return (eb: ExpressionBuilder<any, any>) =>
+         eb
+            .selectFrom(`${self.entity.name} as ${relationRef}`)
+            .whereRef(entityRef, "=", otherRef)
+            .$if(self.cardinality === 1, (qb) => qb.limit(1));
    }
 
    /**