forked from pinks/eris
1
0
Fork 0

webhooks #1

Merged
Akiru merged 12 commits from webhooks into main 2024-01-26 13:20:20 +00:00
8 changed files with 159 additions and 42 deletions

View File

@ -1,17 +1,12 @@
# Eris the Bot # Nyx the Bot
Fork of Eris the Bot https://eris.lisq.eu
[![Website](https://img.shields.io/website?url=https%3A%2F%2Feris.lisq.eu%2F)](https://eris.lisq.eu/)
![Unique users](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Feris.lisq.eu%2Fapi%2Fstats&query=%24.userCount&label=unique%20users)
![Generated images](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Feris.lisq.eu%2Fapi%2Fstats&query=%24.imageCount&label=images%20generated)
![Processed steps](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Feris.lisq.eu%2Fapi%2Fstats&query=%24.stepCount&label=steps%20processed)
![Painted pixels](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Feris.lisq.eu%2Fapi%2Fstats&query=%24.pixelCount&label=pixels%20painted)
Telegram bot for generating images from text. Telegram bot for generating images from text.
## Requirements ## Requirements
- [Deno](https://deno.land/) - [Deno](https://deno.land/) (for the bot server)
- [Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui/) - [Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui/) (for the worker that generates images)
## Options ## Options
@ -20,17 +15,23 @@ You can put these in `.env` file or pass them as environment variables.
- `TG_BOT_TOKEN` - Telegram bot token. Get yours from [@BotFather](https://t.me/BotFather). - `TG_BOT_TOKEN` - Telegram bot token. Get yours from [@BotFather](https://t.me/BotFather).
Required. Required.
- `DENO_KV_PATH` - [Deno KV](https://deno.land/api?s=Deno.openKv&unstable) database file path. A - `DENO_KV_PATH` - [Deno KV](https://deno.land/api?s=Deno.openKv&unstable) database file path. A
temporary file is used by default. temporary file is used by default. Example: /opt/data/botdata.kv
- `LOG_LEVEL` - [Log level](https://deno.land/std@0.201.0/log/mod.ts?s=LogLevels). Default: `INFO`. - `LOG_LEVEL` - [Log level](https://deno.land/std@0.201.0/log/mod.ts?s=LogLevels). Default: `INFO`.
## Running ## Running
1. Start Eris: `deno task start` 1. Start Eris: `deno task start`
2. Visit [Eris WebUI](http://localhost:5999/) and login via Telegram. 2. Visit [Eris WebUI](http://localhost:8443/) and login via Telegram.
3. Promote yourself to admin in the Eris WebUI. 3. Promote yourself to admin in the Eris WebUI.
4. Start Stable Diffusion WebUI: `./webui.sh --api` (in SD WebUI directory) 4. Start Stable Diffusion WebUI: `./webui.sh --api` (in SD WebUI directory)
5. Add a new worker in the Eris WebUI. 5. Add a new worker in the Eris WebUI.
## This fork requires the use of webhooks.
1. You need a reverse Proxy and HTTPS certificate set up that proxies all requests from a domain on port 443 (e.g. nyx.akiru.de) to the backend of this bot (:8443).
2. Change the Webhook URL to your own one in /mod/bot.ts - There are also console log statements you can uncomment for troubleshooting.
3. Make sure the DNS and firewall are set up for the webhook to be reachable, because otherwise the bot fails to start.
## Codegen ## Codegen
The Stable Diffusion API types are auto-generated. To regenerate them, first start your SD WebUI The Stable Diffusion API types are auto-generated. To regenerate them, first start your SD WebUI
@ -38,8 +39,8 @@ with `--nowebui --api`, and then run `deno task generate`
## Project structure ## Project structure
- `/api` - Eris API served at `http://localhost:5999/api/`. - `/api` - Eris API served at `http://localhost:8443/api/`.
- `/app` - Queue handling and other core processes. - `/app` - Queue handling and other core processes.
- `/bot` - Handling bot commands and other updates from Telegram API. - `/bot` - Handling bot commands and other updates from Telegram API.
- `/ui` - Eris WebUI frontend files served at `http://localhost:5999/`. - `/ui` - Eris WebUI frontend files served at `http://localhost:8443/`.
- `/util` - Utility functions shared by other parts. - `/util` - Utility functions shared by other parts.

View File

@ -2,11 +2,15 @@ import { route } from "reroute";
import { serveSpa } from "serve_spa"; import { serveSpa } from "serve_spa";
import { api } from "./serveApi.ts"; import { api } from "./serveApi.ts";
import { fromFileUrl } from "std/path/mod.ts"; import { fromFileUrl } from "std/path/mod.ts";
// New function that handles the webhook
import { handleWebhook } from "../bot/mod.ts";
export async function serveUi() { export async function serveUi() {
const server = Deno.serve({ port: 5999 }, (request) => const server = Deno.serve({ port: 8443 }, (request) =>
route(request, { route(request, {
"/api/*": (request) => api.fetch(request), "/api/*": (request) => api.fetch(request),
"/webhook": handleWebhook, // Create the webhook route handle
"/*": (request) => "/*": (request) =>
serveSpa(request, { serveSpa(request, {
fsRoot: fromFileUrl(new URL("../ui/", import.meta.url)), fsRoot: fromFileUrl(new URL("../ui/", import.meta.url)),

View File

@ -1,7 +1,7 @@
import { Api, Bot, Context, RawApi, session, SessionFlavor } from "grammy"; import { Api, Bot, Context, RawApi, session, SessionFlavor } from "grammy";
import { FileFlavor, hydrateFiles } from "grammy_files"; import { FileFlavor, hydrateFiles } from "grammy_files";
import { hydrateReply, ParseModeFlavor } from "grammy_parse_mode"; import { hydrateReply, ParseModeFlavor } from "grammy_parse_mode";
import { run, sequentialize } from "grammy_runner"; import { sequentialize } from "grammy_runner";
import { error, info, warning } from "std/log/mod.ts"; import { error, info, warning } from "std/log/mod.ts";
import { sessions } from "../api/sessionsRoute.ts"; import { sessions } from "../api/sessionsRoute.ts";
import { formatUserChat } from "../utils/formatUserChat.ts"; import { formatUserChat } from "../utils/formatUserChat.ts";
@ -12,6 +12,16 @@ import { img2imgCommand, img2imgQuestion } from "./img2imgCommand.ts";
import { pnginfoCommand, pnginfoQuestion } from "./pnginfoCommand.ts"; import { pnginfoCommand, pnginfoQuestion } from "./pnginfoCommand.ts";
import { queueCommand } from "./queueCommand.ts"; import { queueCommand } from "./queueCommand.ts";
import { txt2imgCommand, txt2imgQuestion } from "./txt2imgCommand.ts"; import { txt2imgCommand, txt2imgQuestion } from "./txt2imgCommand.ts";
import { setConfig, getConfig } from "../app/config.ts";
// Set the new configuration
await setConfig({ maxUserJobs: 1, maxJobs: 500 });
// Fetch the updated configuration
const updatedConfig = await getConfig();
// Log the updated configuration to the console
console.log("Updated Configuration:", updatedConfig);
interface SessionData { interface SessionData {
chat: ErisChatData; chat: ErisChatData;
@ -107,18 +117,38 @@ bot.use(async (ctx, next) => {
} }
}); });
bot.api.setMyShortDescription("I can generate furry images from text"); // Wrap the calls in try-catch for error handling
bot.api.setMyDescription( async function setupBotCommands() {
"I can generate furry images from text. " + try {
await bot.api.setMyShortDescription("Generate furry images in '704x704', '576x832', '832x576'. https://ko-fi.com/nyxthebot https://nyx.akiru.de/");
} catch (err) {
error(`Failed to set short description: ${err.message}`);
}
try {
await bot.api.setMyDescription(
"I can generate furry images from text. If you want a different size, use 'Size: 576x832' for example." +
"Send /txt2img to generate an image.", "Send /txt2img to generate an image.",
); );
bot.api.setMyCommands([ } catch (err) {
error(`Failed to set description: ${err.message}`);
}
try {
await bot.api.setMyCommands([
{ command: "txt2img", description: "Generate image from text" }, { command: "txt2img", description: "Generate image from text" },
{ command: "img2img", description: "Generate image from image" }, { command: "img2img", description: "Generate image from image" },
{ command: "pnginfo", description: "Try to extract prompt from raw file" }, { command: "pnginfo", description: "Try to extract prompt from raw file" },
{ command: "queue", description: "Show the current queue" }, { command: "queue", description: "Show the current queue" },
{ command: "cancel", description: "Cancel all your requests" }, { command: "cancel", description: "Cancel all your requests" },
]); ]);
} catch (err) {
error(`Failed to set commands: ${err.message}`);
}
}
// Call the setup function
setupBotCommands();
bot.command("start", async (ctx) => { bot.command("start", async (ctx) => {
if (ctx.match) { if (ctx.match) {
@ -173,7 +203,26 @@ bot.command("crash", () => {
throw new Error("Crash command used"); throw new Error("Crash command used");
}); });
export async function runBot() { // Set up the webhook in the telegram API and initialize the bot
const runner = run(bot, { runner: { silent: true } }); await bot.api.setWebhook('https://nyx.akiru.de/webhook');
await runner.task(); await bot.init();
// Function to handle incoming webhook requests
export async function handleWebhook(req: Request): Promise<Response> {
try {
const body = await req.json();
// console.log("Received webhook data:", JSON.stringify(body, null, 2));
// Log before processing update
// console.log("Processing update through handleUpdate...");
await bot.handleUpdate(body); // Process the update
// Log after processing update
// console.log("Update processed successfully.");
return new Response(JSON.stringify({ status: 'ok' }), { headers: { 'Content-Type': 'application/json' } });
} catch (error) {
// Detailed error logging
console.error("Error in handleWebhook:", error);
return new Response("Error", { status: 500 });
}
} }

View File

@ -4,7 +4,9 @@ import { ConsoleHandler } from "std/log/handlers.ts";
import { LevelName, setup } from "std/log/mod.ts"; import { LevelName, setup } from "std/log/mod.ts";
import { serveUi } from "./api/mod.ts"; import { serveUi } from "./api/mod.ts";
import { runAllTasks } from "./app/mod.ts"; import { runAllTasks } from "./app/mod.ts";
import { runBot } from "./bot/mod.ts"; // runbot shouldn't be needed for webhooks?
// import { runBot } from "./bot/mod.ts";
import "./bot/mod.ts";
const logLevel = Deno.env.get("LOG_LEVEL")?.toUpperCase() as LevelName ?? "INFO"; const logLevel = Deno.env.get("LOG_LEVEL")?.toUpperCase() as LevelName ?? "INFO";
@ -20,7 +22,8 @@ setup({
// run parts of the app // run parts of the app
await Promise.all([ await Promise.all([
runBot(), // runbot shouldn't be needed for webhooks?
// runBot(),
runAllTasks(), runAllTasks(),
serveUi(), serveUi(),
]); ]);

1
monitoring/Readme.md Normal file
View File

@ -0,0 +1 @@
These scripts exist to automatically restart the bot in case of errors and connectivity issues. They should either be run as systemd services or cronjobs.

29
monitoring/monitoring.sh Normal file
View File

@ -0,0 +1,29 @@
#!/bin/bash
# This script has to be run as systemd service. It checks journalctl output for errors and restarts the bot automatically when they occur too often.
# Define the regex pattern to search for occurrences of "GrammyError" and "sendMediaGroup"
ERROR_PATTERN="GrammyError.*sendMediaGroup"
# Initialize the counter for consecutive occurrences
error_count=0
# Monitor the journalctl output
journalctl -xe -f | while read line; do
# Check if the line contains the pattern
if echo "$line" | grep -qE "$ERROR_PATTERN"; then
# Increment the error counter
((error_count++))
# Check if the error has occurred 4 times in a row
if [ $error_count -eq 4 ]; then
# Restart the bot service
systemctl restart nyxthebot
# Reset the error counter
error_count=0
fi
else
# Reset the error counter if the line does not contain the error
error_count=0
fi
done

30
monitoring/sleepy.sh Normal file
View File

@ -0,0 +1,30 @@
#!/bin/bash
# When the bot gets eepy sleepy, this script will restart it. It happens, when no jobs come in for multiple minutes.
# The script interfaces with the REST API of the bot and checks if the image count has changed.
# Run this script as cronjob all 4-5 minutes.
# Define the URL and the stats file path
URL="https://YOUR_URL/api/stats"
STATS_FILE="YOUR_FILE_PATH/stats.txt"
# Fetch the data and extract imageCount
imageCount=$(curl -s "$URL" | jq '.imageCount')
# Check if stats file exists and read the last value
if [ -f "$STATS_FILE" ]; then
lastValue=$(tail -n 1 "$STATS_FILE")
else
lastValue=""
fi
# Save the new value to the file
echo "$imageCount" >> "$STATS_FILE"
# Keep only the last 2 values in the file
tail -n 2 "$STATS_FILE" > "$STATS_FILE.tmp" && mv "$STATS_FILE.tmp" "$STATS_FILE"
# Compare the two values and restart the service if they are the same
if [ "$lastValue" == "$imageCount" ]; then
systemctl restart nyxthebot
fi

View File

@ -7,7 +7,7 @@ export function StatsPage() {
const getGlobalStats = useSWR( const getGlobalStats = useSWR(
["/stats", {}] as const, ["/stats", {}] as const,
(args) => fetchApi(...args).then(handleResponse), (args) => fetchApi(...args).then(handleResponse),
{ refreshInterval: 2_000 }, { refreshInterval: 1_00 },
); );
return ( return (
@ -18,13 +18,13 @@ export function StatsPage() {
className="font-bold text-zinc-700 dark:text-zinc-300" className="font-bold text-zinc-700 dark:text-zinc-300"
value={getGlobalStats.data?.pixelStepCount ?? 0} value={getGlobalStats.data?.pixelStepCount ?? 0}
digits={15} digits={15}
transitionDurationMs={3_000} transitionDurationMs={1_00}
/> />
<Counter <Counter
className="text-base" className="text-base"
value={(getGlobalStats.data?.pixelStepsPerMinute ?? 0) / 60} value={(getGlobalStats.data?.pixelStepsPerMinute ?? 0) / 60}
digits={9} digits={9}
transitionDurationMs={2_000} transitionDurationMs={1_00}
postfix="/s" postfix="/s"
/> />
</p> </p>
@ -34,13 +34,13 @@ export function StatsPage() {
className="font-bold text-zinc-700 dark:text-zinc-300" className="font-bold text-zinc-700 dark:text-zinc-300"
value={getGlobalStats.data?.pixelCount ?? 0} value={getGlobalStats.data?.pixelCount ?? 0}
digits={15} digits={15}
transitionDurationMs={3_000} transitionDurationMs={1_00}
/> />
<Counter <Counter
className="text-base" className="text-base"
value={(getGlobalStats.data?.pixelsPerMinute ?? 0) / 60} value={(getGlobalStats.data?.pixelsPerMinute ?? 0) / 60}
digits={9} digits={9}
transitionDurationMs={2_000} transitionDurationMs={1_00}
postfix="/s" postfix="/s"
/> />
</p> </p>
@ -51,14 +51,14 @@ export function StatsPage() {
className="font-bold text-zinc-700 dark:text-zinc-300" className="font-bold text-zinc-700 dark:text-zinc-300"
value={getGlobalStats.data?.stepCount ?? 0} value={getGlobalStats.data?.stepCount ?? 0}
digits={9} digits={9}
transitionDurationMs={3_000} transitionDurationMs={1_00}
/> />
<Counter <Counter
className="text-base" className="text-base"
value={(getGlobalStats.data?.stepsPerMinute ?? 0) / 60} value={(getGlobalStats.data?.stepsPerMinute ?? 0) / 60}
digits={3} digits={3}
fractionDigits={3} fractionDigits={3}
transitionDurationMs={2_000} transitionDurationMs={1_00}
postfix="/s" postfix="/s"
/> />
</p> </p>
@ -68,14 +68,14 @@ export function StatsPage() {
className="font-bold text-zinc-700 dark:text-zinc-300" className="font-bold text-zinc-700 dark:text-zinc-300"
value={getGlobalStats.data?.imageCount ?? 0} value={getGlobalStats.data?.imageCount ?? 0}
digits={9} digits={9}
transitionDurationMs={3_000} transitionDurationMs={1_00}
/> />
<Counter <Counter
className="text-base" className="text-base"
value={(getGlobalStats.data?.imagesPerMinute ?? 0) / 60} value={(getGlobalStats.data?.imagesPerMinute ?? 0) / 60}
digits={3} digits={3}
fractionDigits={3} fractionDigits={3}
transitionDurationMs={2_000} transitionDurationMs={1_00}
postfix="/s" postfix="/s"
/> />
</p> </p>
@ -86,7 +86,7 @@ export function StatsPage() {
className="font-bold text-zinc-700 dark:text-zinc-300" className="font-bold text-zinc-700 dark:text-zinc-300"
value={getGlobalStats.data?.userCount ?? 0} value={getGlobalStats.data?.userCount ?? 0}
digits={6} digits={6}
transitionDurationMs={1_500} transitionDurationMs={1_00}
/> />
</p> </p>
</div> </div>