shishantbiswas/aniwatch-api
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #80 from ghoshRitesh12/cache

Browse Source 
Add caching layer for improved performance
This commit is contained in:
Ritesh Ghosh
2024-12-07 21:17:27 +05:30
committed by GitHub
parent 03771a2096 02c8d107dc
commit b0718180a0
7 changed files with 204 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+8 -6
View File
@@ -2,7 +2,7 @@
ANIWATCH_API_PORT=4000
# env to control allowed origins
ANIWATCH_API_CORS_ALLOWED_ORIGINS=https://your-production-domain.com,https://another-trusted-domain.com
ANIWATCH_API_CORS_ALLOWED_ORIGINS=<https://your-production-domain.com,https://another-trusted-domain.com>
# RATE LIMIT
# duration to track requests (in milliseconds) for rate limiting. here, 30*60*1000 = 1800000 = 30 minutes
@@ -12,12 +12,14 @@ ANIWATCH_API_MAX_REQS=70
# CAUTION:
# For personal deployments, if you wanna have rate limitting
# in your application, then set this env to your deployed
# instance's hostname, otherwise don't set or have this env at all.
# If you set this env to an incorrect value, you may face other issues.
# in your application, then set the env below to your deployed
# instance's hostname, otherwise don't set or have it at all.
# If you set the env below to an incorrect value, you may face other issues.
# ANIWATCH_API_HOSTNAME="api-aniwatch.onrender.com"
# NOTE: this env is "required" for vercel deployments
# ANIWATCH_VERCEL_DEPLOYMENT=<true or any non zero value>
# ANIWATCH_API_VERCEL_DEPLOYMENT=<true or any non zero value>
# env to use optional redis caching functionality
ANIWATCH_API_REDIS_CONN_URL=<rediss://default:your-secure-password@your-redis-instance-name.provider.com:6379>
README.md
+36 -24
View File
@@ -14,9 +14,11 @@
# <p align="center">Aniwatch API</p>
<div align="center">
A free restful API serving anime information from <a href="https://hianime.to" target="_blank">hianime.to</a>
<br/><br/>
<strong>
A free RESTful API serving anime information from <a href="https://hianime.to" target="_blank">hianime.to</a>
<br/>
<div>
<a
href="https://github.com/ghoshRitesh12/aniwatch-api/issues/new?assignees=ghoshRitesh12&labels=bug&template=bug-report.yml"
>
@@ -28,7 +30,7 @@
>
Feature request
</a>
</strong>
</div>
</div>
<br/>
@@ -56,17 +58,19 @@
> [!IMPORTANT]
>
> 1. [https://api-aniwatch.onrender.com](https://api-aniwatch.onrender.com/) is only meant to demo the API and has rate-limiting enabled to minimise bandwidth consumption. It is recommended to deploy your own instance for personal use by customizing the api as you need it to be.
> 2. This API is just an unofficial api for [hianime.to](https://hianime.to) and is in no other way officially related to the same.
> 3. The content that this api provides is not mine, nor is it hosted by me. These belong to their respective owners. This api just demonstrates how to build an api that scrapes websites and uses their content.
> 1. [https://api-aniwatch.onrender.com](https://api-aniwatch.onrender.com/) is only meant to demo the API and has rate-limiting enabled to minimize bandwidth consumption. It is recommended to deploy your own instance for personal use by customizing the API as you need it to be.
> 2. This API is just an unofficial API for [hianime.to](https://hianime.to) and is in no other way officially related to the same.
> 3. The content that this API provides is not mine, nor is it hosted by me. These belong to their respective owners. This API just demonstrates how to build an API that scrapes websites and uses their content.
## Table of Contents
- [Installation](#installation)
- [Local](#local)
- [Docker](#docker)
- [Envs](#envs)
- [Host your instance](#-host-your-instance)
- [Configuration](#️configuration)
- [Custom HTTP Headers](#custom-http-headers)
- [Environment Variables](#environment-variables)
- [Host your instance](#host-your-instance)
- [Vercel](#vercel)
- [Render](#render)
- [Documentation](#documentation)
@@ -115,29 +119,36 @@
### Docker
Docker image is available at [GitHub Container Registry](https://github.com/ghoshRitesh12/aniwatch-api/pkgs/container/aniwatch).
The Docker image is available at [The GitHub Container Registry](https://github.com/ghoshRitesh12/aniwatch-api/pkgs/container/aniwatch).
Run the following commands to pull and run the docker image.
```bash
docker pull ghcr.io/ghoshritesh12/aniwatch
docker run -p 4000:4000 ghcr.io/ghoshritesh12/aniwatch
docker run -d --name aniwatch-api -p 4000:4000 ghcr.io/ghoshritesh12/aniwatch
```
The above command will start the server on port 4000. You can access the server at [http://localhost:4000](http://localhost:4000) and you can also change the port by changing the `-p` option to `-p <port>:4000`.
The above command will start the server on port 4000. You can access the server at [http://localhost:4000](http://localhost:4000), and you can also change the port by changing the `-p` option to `-p <port>:4000`.
You can also add the `-d` flag to run the container in detached mode.
The `-d` flag runs the container in detached mode, and the `--name` flag is used to name the container that's about to run.
## <span id="envs">⚙️ Envs</span>
## <span id="configuration">⚙️ Configuration</span>
More info can be found in [`.env.example`](https://github.com/ghoshRitesh12/aniwatch-api/blob/main/.env.example) file
### Custom HTTP Headers
Currently this API supports parsing of only one custom header, and more may be implemented in the future to accommodate varying needs.
- `ANIWATCH_API_PORT`: port number of the aniwatch api
- `ANIWATCH_API_WINDOW_MS`: duration to track requests for rate limitting (in milliseconds)
- `ANIWATCH_API_MAX_REQS`: maximum number of requests in the `ANIWATCH_API_WINDOW_MS` timeperiod
- `ANIWATCH_API_CORS_ALLOWED_ORIGINS`: allowed origins, separated by commas and no spaces in between
- `ANIWATCH_VERCEL_DEPLOYMENT`: required for distinguishing vercel deployment from other ones, set it to true of any other non-zero value
- `ANIWATCH_API_HOSTNAME`: set this to your api instance's hostname to enable rate limitting, don't have this value if you don't wish to rate limit
- `X-ANIWATCH-CACHE-EXPIRY`: this custom header is used to specify the cache expiration duration in **seconds** (defaults to 60 if the header is missing). The `ANIWATCH_API_REDIS_CONN_URL` env is required for this custom header to function as intended; otherwise, there's no point in setting this custom header.
### Environment Variables
More info can be found in [`.env.example`](https://github.com/ghoshRitesh12/aniwatch-api/blob/main/.env.example) file, where envs's having a value that is contained within `<` `>` angled brackets are just examples and should be replaced with relevant ones.
- `ANIWATCH_API_PORT`: port number of the aniwatch API.
- `ANIWATCH_API_WINDOW_MS`: duration to track requests for rate limiting (in milliseconds).
- `ANIWATCH_API_MAX_REQS`: maximum number of requests in the `ANIWATCH_API_WINDOW_MS` time period.
- `ANIWATCH_API_CORS_ALLOWED_ORIGINS`: allowed origins, separated by commas and no spaces in between.
- `ANIWATCH_API_VERCEL_DEPLOYMENT`: required for distinguishing Vercel deployment from other ones; set it to true or any other non-zero value.
- `ANIWATCH_API_HOSTNAME`: set this to your api instance's hostname to enable rate limiting, don't have this value if you don't wish to rate limit.
- `ANIWATCH_API_REDIS_CONN_URL`: this env is optional by default and can be set to utilize Redis caching functionality. It has to be a valid connection URL; otherwise, the Redis client can throw unexpected errors.
## <span id="host-your-instance">⛅ Host your instance</span>
@@ -145,7 +156,8 @@ More info can be found in [`.env.example`](https://github.com/ghoshRitesh12/aniw
>
> For personal deployments:
>
> - If you wanna have rate limitting in your application, then set the `ANIWATCH_API_HOSTNAME` env to your deployed instance's hostname, otherwise don't set or have this env at all. If you set this env to an incorrect value, you may face other issues.
> - If you want to have rate limiting in your application, then set the `ANIWATCH_API_HOSTNAME` env to your deployed instance's hostname; otherwise, don't set or have this env at all. If you set this env to an incorrect value, you may face other issues.
> - It's optional by default, but if you want to have endpoint response caching functionality, then set the `ANIWATCH_API_REDIS_CONN_URL` env to a valid Redis connection URL. If the connection URL is invalid, the Redis client can throw unexpected errors.
> - Remove the if block from the [`server.ts`](https://github.com/ghoshRitesh12/aniwatch-api/blob/main/src/server.ts) file, spanning from lines [71](https://github.com/ghoshRitesh12/aniwatch-api/blob/main/src/server.ts#L71) to [83](https://github.com/ghoshRitesh12/aniwatch-api/blob/main/src/server.ts#L83).
### Vercel
@@ -156,7 +168,7 @@ Deploy your own instance of Aniwatch API on Vercel.
> [!NOTE]
>
> When deploying to vercel, set an env named `ANIWATCH_VERCEL_DEPLOYMENT` to `true` or any non-zero value, but this env must be present.
> When deploying to vercel, set an env named `ANIWATCH_API_VERCEL_DEPLOYMENT` to `true` or any non-zero value, but this env must be present.
### Render
package.json
+2 -1
View File
@@ -37,7 +37,8 @@
"aniwatch": "^2.4.1",
"dotenv": "^16.4.5",
"hono": "^4.6.3",
"hono-rate-limiter": "^0.4.0"
"hono-rate-limiter": "^0.4.0",
"ioredis": "^5.4.1"
},
"devDependencies": {
"@types/node": "^22.7.4",
src/config/cache.ts
+50
View File
@@ -0,0 +1,50 @@
import { config } from "dotenv";
import { Redis } from "ioredis";
config();
export class AniwatchAPICache {
private _client: Redis | null;
public isOptional: boolean = true;
static DEFAULT_CACHE_EXPIRY_SECONDS = 60 as const;
static CACHE_EXPIRY_HEADER_NAME = "X-ANIWATCH-CACHE-EXPIRY" as const;
constructor() {
const redisConnURL = process.env?.ANIWATCH_API_REDIS_CONN_URL;
this.isOptional = !Boolean(redisConnURL);
this._client = this.isOptional ? null : new Redis(String(redisConnURL));
}
set(key: string | Buffer, value: string | Buffer | number) {
if (this.isOptional) return;
return this._client?.set(key, value);
}
get(key: string | Buffer) {
if (this.isOptional) return;
return this._client?.get(key);
}
/**
* @param expirySeconds set to 60 by default
*/
async getOrSet<T>(
key: string | Buffer,
setCB: () => Promise<T>,
expirySeconds: number = AniwatchAPICache.DEFAULT_CACHE_EXPIRY_SECONDS
) {
const cachedData = this.isOptional
? null
: (await this._client?.get(key)) || null;
let data = JSON.parse(String(cachedData)) as T;
if (!data) {
data = await setCB();
await this._client?.set(key, JSON.stringify(data), "EX", expirySeconds);
}
return data;
}
}
export const cache = new AniwatchAPICache();
src/config/variables.ts
+8
View File
@@ -0,0 +1,8 @@
type CacheVariables = {
CACHE_CONFIG: {
key: string;
duration: number;
};
};
export type AniwatchAPIVariables = {} & CacheVariables;
src/routes/hianime.ts
+80 -17
View File
@@ -1,123 +1,186 @@
import { Hono } from "hono";
import { HiAnime } from "aniwatch";
import { cache } from "../config/cache.js";
import type { AniwatchAPIVariables } from "../config/variables.js";
const hianime = new HiAnime.Scraper();
const hianimeRouter = new Hono();
const hianimeRouter = new Hono<{ Variables: AniwatchAPIVariables }>();
// /api/v2/hianime
hianimeRouter.get("/", (c) => c.redirect("/", 301));
// /api/v2/hianime/home
hianimeRouter.get("/home", async (c) => {
const data = await hianime.getHomePage();
const cacheConfig = c.get("CACHE_CONFIG");
const data = await cache.getOrSet<HiAnime.ScrapedHomePage>(
cacheConfig.key,
hianime.getHomePage,
cacheConfig.duration
);
return c.json({ success: true, data }, { status: 200 });
});
// /api/v2/hianime/category/{name}?page={page}
hianimeRouter.get("/category/:name", async (c) => {
const cacheConfig = c.get("CACHE_CONFIG");
const categoryName = decodeURIComponent(
c.req.param("name").trim()
) as HiAnime.AnimeCategories;
const page: number =
Number(decodeURIComponent(c.req.query("page") || "")) || 1;
const data = await hianime.getCategoryAnime(categoryName, page);
const data = await cache.getOrSet<HiAnime.ScrapedAnimeCategory>(
cacheConfig.key,
async () => hianime.getCategoryAnime(categoryName, page),
cacheConfig.duration
);
return c.json({ success: true, data }, { status: 200 });
});
// /api/v2/hianime/genre/{name}?page={page}
hianimeRouter.get("/genre/:name", async (c) => {
const cacheConfig = c.get("CACHE_CONFIG");
const genreName = decodeURIComponent(c.req.param("name").trim());
const page: number =
Number(decodeURIComponent(c.req.query("page") || "")) || 1;
const data = await hianime.getGenreAnime(genreName, page);
const data = await cache.getOrSet<HiAnime.ScrapedGenreAnime>(
cacheConfig.key,
async () => hianime.getGenreAnime(genreName, page),
cacheConfig.duration
);
return c.json({ success: true, data }, { status: 200 });
});
// /api/v2/hianime/producer/{name}?page={page}
hianimeRouter.get("/producer/:name", async (c) => {
const cacheConfig = c.get("CACHE_CONFIG");
const producerName = decodeURIComponent(c.req.param("name").trim());
const page: number =
Number(decodeURIComponent(c.req.query("page") || "")) || 1;
const data = await hianime.getProducerAnimes(producerName, page);
const data = await cache.getOrSet<HiAnime.ScrapedProducerAnime>(
cacheConfig.key,
async () => hianime.getProducerAnimes(producerName, page),
cacheConfig.duration
);
return c.json({ success: true, data }, { status: 200 });
});
// /api/v2/hianime/schedule?date={date}
hianimeRouter.get("/schedule", async (c) => {
const cacheConfig = c.get("CACHE_CONFIG");
const date = decodeURIComponent(c.req.query("date") || "");
const data = await hianime.getEstimatedSchedule(date);
const data = await cache.getOrSet<HiAnime.ScrapedEstimatedSchedule>(
cacheConfig.key,
async () => hianime.getEstimatedSchedule(date),
cacheConfig.duration
);
return c.json({ success: true, data }, { status: 200 });
});
// /api/v2/hianime/search?q={query}&page={page}&filters={...filters}
hianimeRouter.get("/search", async (c) => {
const cacheConfig = c.get("CACHE_CONFIG");
let { q: query, page, ...filters } = c.req.query();
query = decodeURIComponent(query || "");
const pageNo = Number(decodeURIComponent(page || "")) || 1;
const data = await hianime.search(query, pageNo, filters);
const data = await cache.getOrSet<HiAnime.ScrapedAnimeSearchResult>(
cacheConfig.key,
async () => hianime.search(query, pageNo, filters),
cacheConfig.duration
);
return c.json({ success: true, data }, { status: 200 });
});
// /api/v2/hianime/search/suggestion?q={query}
hianimeRouter.get("/search/suggestion", async (c) => {
const cacheConfig = c.get("CACHE_CONFIG");
const query = decodeURIComponent(c.req.query("q") || "");
const data = await hianime.searchSuggestions(query);
const data = await cache.getOrSet<HiAnime.ScrapedAnimeSearchSuggestion>(
cacheConfig.key,
async () => hianime.searchSuggestions(query),
cacheConfig.duration
);
return c.json({ success: true, data }, { status: 200 });
});
// /api/v2/hianime/anime/{animeId}
hianimeRouter.get("/anime/:animeId", async (c) => {
const cacheConfig = c.get("CACHE_CONFIG");
const animeId = decodeURIComponent(c.req.param("animeId").trim());
const data = await hianime.getInfo(animeId);
const data = await cache.getOrSet<HiAnime.ScrapedAnimeAboutInfo>(
cacheConfig.key,
async () => hianime.getInfo(animeId),
cacheConfig.duration
);
return c.json({ success: true, data }, { status: 200 });
});
// /api/v2/hianime/episode/servers?animeEpisodeId={id}
hianimeRouter.get("/episode/servers", async (c) => {
const cacheConfig = c.get("CACHE_CONFIG");
const animeEpisodeId = decodeURIComponent(
c.req.query("animeEpisodeId") || ""
);
const data = await hianime.getEpisodeServers(animeEpisodeId);
const data = await cache.getOrSet<HiAnime.ScrapedEpisodeServers>(
cacheConfig.key,
async () => hianime.getEpisodeServers(animeEpisodeId),
cacheConfig.duration
);
return c.json({ success: true, data }, { status: 200 });
});
// episodeId=steinsgate-3?ep=230
// /api/v2/hianime/episode/sources?animeEpisodeId={episodeId}?server={server}&category={category (dub or sub)}
hianimeRouter.get("/episode/sources", async (c) => {
const cacheConfig = c.get("CACHE_CONFIG");
const animeEpisodeId = decodeURIComponent(
c.req.query("animeEpisodeId") || ""
);
const server = decodeURIComponent(
c.req.query("server") || HiAnime.Servers.VidStreaming
) as HiAnime.AnimeServers;
const category = decodeURIComponent(c.req.query("category") || "sub") as
| "sub"
| "dub"
| "raw";
const data = await hianime.getEpisodeSources(
animeEpisodeId,
server,
category
const data = await cache.getOrSet<HiAnime.ScrapedAnimeEpisodesSources>(
cacheConfig.key,
async () => hianime.getEpisodeSources(animeEpisodeId, server, category),
cacheConfig.duration
);
return c.json({ success: true, data }, { status: 200 });
});
// /api/v2/hianime/anime/{anime-id}/episodes
hianimeRouter.get("/anime/:animeId/episodes", async (c) => {
const cacheConfig = c.get("CACHE_CONFIG");
const animeId = decodeURIComponent(c.req.param("animeId").trim());
const data = await hianime.getEpisodes(animeId);
const data = await cache.getOrSet<HiAnime.ScrapedAnimeEpisodes>(
cacheConfig.key,
async () => hianime.getEpisodes(animeId),
cacheConfig.duration
);
return c.json({ success: true, data }, { status: 200 });
});
src/server.ts
+20 -4
View File
@@ -6,12 +6,14 @@ import { ratelimit } from "./config/ratelimit.js";
import { hianimeRouter } from "./routes/hianime.js";
import { serveStatic } from "@hono/node-server/serve-static";
import { serve } from "@hono/node-server";
import { Hono } from "hono";
import { logger } from "hono/logger";
import { serve } from "@hono/node-server";
import { serveStatic } from "@hono/node-server/serve-static";
import { HiAnimeError } from "aniwatch";
import { AniwatchAPICache } from "./config/cache.js";
import type { AniwatchAPIVariables } from "./config/variables.js";
config();
@@ -19,7 +21,7 @@ const BASE_PATH = "/api/v2" as const;
const PORT: number = Number(process.env.ANIWATCH_API_PORT) || 4000;
const ANIWATCH_API_HOSTNAME = process.env?.ANIWATCH_API_HOSTNAME;
const app = new Hono();
const app = new Hono<{ Variables: AniwatchAPIVariables }>();
app.use(logger());
app.use(corsConfig);
@@ -35,6 +37,20 @@ if (ISNT_PERSONAL_DEPLOYMENT) {
app.use("/", serveStatic({ root: "public" }));
app.get("/health", (c) => c.text("OK", { status: 200 }));
app.use(async (c, next) => {
const { pathname, search } = new URL(c.req.url);
c.set("CACHE_CONFIG", {
key: `${pathname.slice(BASE_PATH.length) + search}`,
duration: Number(
c.req.header(AniwatchAPICache.CACHE_EXPIRY_HEADER_NAME) ||
AniwatchAPICache.DEFAULT_CACHE_EXPIRY_SECONDS
),
});
await next();
});
app.basePath(BASE_PATH).route("/hianime", hianimeRouter);
app
.basePath(BASE_PATH)
@@ -57,7 +73,7 @@ app.onError((err, c) => {
});
// NOTE: this env is "required" for vercel deployments
if (!Boolean(process?.env?.ANIWATCH_API_VERCEL_DEPLOYMENT)) {
if (!Boolean(process.env?.ANIWATCH_API_VERCEL_DEPLOYMENT)) {
serve({
port: PORT,
fetch: app.fetch,