nodbot/docs/TESTING.md
2026-10-01 19:42:30 -04:00

28 KiB

NodBot Manual Regression Test Plan

Human-executed regression suite for validating a nodbot-dev build before promoting it to prod. Written against the feature set as of v3.7.0 (tag v*-dev* builds). You are the only tester; every test is written as explicit Discord-side steps plus an expected result.

How to use this document

  • Run the full suite before any prod promotion where the diff touches shared code (main.js, functions.js, CustomModules/*), a dependency bump, or the Dockerfile/CI workflow.
  • Run just the [V14]-tagged tests when the diff is scoped to the discord.js v13→v14 migration itself (see Appendix B for what each tag maps to). Search this file for [V14] to filter.
  • Every test assumes you're logged into your Development Server as yourself, with nodbot-dev online, posting in the playground channel unless a test says otherwise. The status channel is where you confirm startup/alert messages land.
  • "Expected" describes correct behavior. If actual behavior diverges, log it (console/Docker logs + what you saw in Discord) before moving on — don't stop the run unless a test earlier in Section 1 fails, since everything downstream depends on the bot being online.
  • As of this writing, the v13→v14 code migration itself has not landed — this plan is written to be ready the moment it does. Discord-side steps are written generically (click buttons, send messages, read embeds) so they don't need rewriting when the builder/intent code changes underneath them.
  • Message Content Intent portal status (prod and dev apps) was unconfirmed at the time this plan was written — Section 2 treats this as a deploy-blocking check rather than assuming it's fine.

0. Pre-flight

ID Steps Expected
PRE-1 Confirm nodbot-dev container is the newly-built image (docker ps / check image digest or tag against the Gitea Actions run that just completed). Running container matches the dev build you intend to test, not a stale one.
PRE-2 Confirm dev DB connectivity info in the dev container's env matches the intended (stale-but-functional) dev DB, not prod. No risk of the test run writing to prod tables.
PRE-3 [V14] docker logs the dev container from a cold start (restart it if it's been up a while) and read the full boot log. No unhandled exceptions, no UnhandledPromiseRejection, no deprecation warnings about intents/partials being strings instead of enum members (this is the first symptom of a half-finished v14 migration).

1. Startup & Lifecycle [V14 critical]

The ready handler is the single highest-risk piece of this migration — Client construction (intents/partials), collection hydration, two external API calls, and the startup status post all happen here before anything else can be tested.

ID Steps Expected
START-1 [V14] Restart nodbot-dev from a clean state. Watch docker logs -f. Process starts, logs Ready!, no crash. On v13 this uses string-literal intents/partials ('GUILDS', 'CHANNEL'); on v14 these must be GatewayIntentBits/Partials enum members — a leftover string will throw at Client construction (RangeError or similar) and the process won't even reach ready.
START-2 Check the status channel. A message posts: <timestamp> -- @<ownerId>\nStartup Sequence Complete. The owner mention must resolve to a real ping, not literal <@undefined>.
START-3 In the dev console/log output, confirm each collection-build log line appears (Interaction Storage, Slash Commands, Dot Commands, Valid Commands, Medical Advice/Roaches, Memberships Collection Built) — only visible if isDev=true. All six collection-builder log lines present, no errors between them.
START-4 Confirm GIF/pasta/joint/request/strain/medical-advice DB downloads complete (no thrown DB errors in logs). No throw err stack traces from any functions.download.* call.
START-5 Confirm the METAR and D-ATIS ICAO master lists fetch successfully (watch for An error occurred during the HTTP request or decompression errors in logs). Both external calls succeed; no caught/logged errors.
START-6 Immediately after Startup Sequence Complete posts, check for a "Guild Check:" log block. One line per guild the bot is in; the banned-server guild (1224396616555823124, if the dev bot is ever added to it) would trigger a leave + alert — otherwise no action.
START-7 [V14] Add nodbot-dev to a throwaway/second test server (or remove and re-add it to the Dev Server if you don't have a spare). A guildCreate message posts to the status channel: I've been added to a new guild: <name> (<id>), followed by a repeat of the banned-server check log block.
START-8 [V14] Remove the bot from that test server. A guildDelete message posts to the status channel: I've been removed from a guild: <name> (<id>).

2. Intents & Message Content [V14 critical]

The client currently requests GUILDS, GUILD_MESSAGES, GUILD_MESSAGE_REACTIONS, DIRECT_MESSAGES, DIRECT_MESSAGE_REACTIONS and partials CHANNEL, MESSAGE — but never explicitly requests the Message Content intent, which every dot-command depends on. This currently works only because small bots (under 100 guilds) are grandfathered by Discord to still receive message.content without the intent. This exemption does not change in v14 — but a v14 rewrite is exactly the kind of change where someone "cleans up" the intents list and accidentally omits something that was previously working by exemption, or where the dev/prod app's actual guild count has crept toward 100 without anyone noticing.

ID Steps Expected
INTENT-1 [V14] — deploy-blocking In the playground channel, send a plain message consisting of just a known gif name + .gif (e.g. test.gif, assuming a test entry exists — else use any real saved name). Bot replies with the GIF URL. If message.content were empty (intent misconfigured), commandData.args/command would be empty/garbage and this would either silently do nothing or throw.
INTENT-2 [V14] Open the Discord Developer Portal for both the prod and nodbot-dev applications → Bot → Privileged Gateway Intents. Record whether "Message Content Intent" is toggled on for each. Document actual state in this test's result (this was unconfirmed when this plan was written). If off for either app and that app's guild count is near/over 100, dot-commands will start silently failing in production with no code change required to trigger it — flag this as a standing operational risk regardless of today's pass/fail.
INTENT-3 Send a DM to nodbot-dev with a dot-command (e.g. .lenny-style content — note .lenny is slash-only, use any dot-command). Confirms DIRECT_MESSAGES intent still functions; bot responds in DM.
INTENT-4 In the playground channel, react to any message the bot can see (reaction-based features aren't implemented today, so this is a smoke test only). No crash, no unexpected behavior — confirms GUILD_MESSAGE_REACTIONS intent doesn't break anything even though nothing currently consumes it.

3. Dot-commands (plain-message name.command triggers)

For every dot-command test below, also confirm the nested-command parser still works: wrap the command in {}, [], or () inside a longer sentence (e.g. yo check this out [test.gif] not kidding) and confirm it still resolves to the same result as the bare form. Run the nested form once per command family (gif, pasta, joint) rather than for every alias.

ID Steps Expected
GIF-1 Send <existing-saved-name>.gif for a name already in the gifs table. Bot replies with the saved GIF URL, no GIPHY call (check logs — no outbound GIPHY request).
GIF-2 Send <some-novel-query>.gif for a name NOT in the gifs table. Bot calls GIPHY, replies with a found GIF URL (or "Sorry I was unable to find a GIF of ..." if GIPHY returns nothing).
GIF-3 Reply to someone else's message with <name>.gif. The GIF reply is sent to the original referenced message, not as a fresh reply to your message.
GIF-4 [V14] Send test.jpg, test.webm, test.mp3 (pick 2-3 aliases) against a saved name. Each alias resolves identically to .gif — confirms CommandData.validate()'s alias-matching loop (dotCommands Collection iteration) still works under v14's Collection import path (discord.js re-exports Collection the same way in v14, but confirm no import path changed).
GIF-5 Embed a nested gif command: before [test.gif] after. Resolves test.gif and ignores the trailing-period fallback.
JOINT-1 Send .joint repeatedly (5-10 times) in the playground channel. Random weed-themed phrase + joint emoji each time; no repeats until the "roach" pool is 85% exhausted, at which point the pool clears.
JOINT-2 Send one of the drink aliases, e.g. .beer or big doinks.joint-style (big doinks is a two-word alias — send literally big doinks.joint). Same behavior as .joint.
METAR-1 Send KBOS.metar (or any ICAO present in config.icaoIds after startup). Bot replies with raw METAR text + a formatted embed (observation time, temp, winds, visibility, clouds, altimeter).
METAR-2 Send kbos, kjfk.metar (multi-ICAO, mixed case/delimiters). Both ICAOs resolve; one reply per ICAO.
METAR-3 Send ZZZZ.metar (invalid/unknown ICAO). Bot replies METAR Error: Invalid ICAO ID Detected: ZZZZ (or similar), no crash.
DATIS-1 Send KBOS.datis (or .atis alias) for an ICAO with known D-ATIS coverage. Bot replies with a formatted D-ATIS embed.
DATIS-2 Send a 3-letter or 5-letter string .datis. Bot replies D-ATIS Error: Invalid ICAO ID. Provide only one ICAO code at a time like KBOS.
DATIS-3 Send a valid-looking but unsupported ICAO .datis. Bot replies "No D-ATIS available for the specified ICAO ID."
PASTA-1 Send <existing-pasta-name>.pasta. Embed reply with the pasta's saved content, author footer, and thumbnail.
PASTA-2 Send <nonexistent-name>.pasta. Embed reply: "Sorry, I couldn't find that pasta."
MD-1 Send .md. Bot replies with a random medical-advice string from the medical_advice table.
REQUEST-1 Send please add more gifs.request. Bot replies with an embed confirming submission; row appears in the requests table with status = Active, correct author, and message.url.
SB-1 Reply to any message with .sb (no extra args). The replied-to message's content gets SpongeBob-cased and sent as a reply to that same message; your invoking .sb message is deleted afterward.
SB-2 Reply to a message with custom text here.sb. The custom text (not the original message content) gets SpongeBob-cased and used as the reply; invoking message deleted.
SB-3 Send a bare some phrase.spongebob with no reply context. Bot sends the SpongeBob-cased phrase as a new channel message (not a reply) and deletes your invoking message.
SB-4 Confirm i and I always render lowercase i, and l/L always render uppercase L, regardless of position. Output matches that rule exactly (this is a deterministic quirk of fn.spongebob, not random — verify it didn't regress).
AUTO-1 Send a message containing both "big" and "doinks" anywhere in the text. Bot replies with one of the 3 bigDoinks responses.
AUTO-2 Send a message containing "ligma" anywhere. Bot replies with one of the ligma responses.
AUTO-3 Send a message containing an ong-triggering keyword. Bot replies with one of the ong responses.
AUTO-4 Send a message containing all fuckYou keywords. Bot replies with one of the fuckYou responses.
AUTO-5 Send a message that is simultaneously a valid dot-command AND contains an autoresponse keyword (e.g. craft a gif name that also contains "ligma"). Both fire: the autoresponse reply and the dot-command reply, independently (confirms they're not mutually exclusive in messageCreate).
EDGE-1 Send a message starting with http or www that also happens to end in .gif-like text (e.g. https://example.com/thing.gif). Not treated as a dot-command (explicit guard in CommandData.validate()). No bot reply as a command.
EDGE-2 Send a message with no period at all. isCommand is false, no dot-command processing attempted, no crash.

4. Slash commands

ID Steps Expected
SLASH-PING /ping Ephemeral Pong!.
SLASH-STATIC /jenny, /lenny, /truth Each returns its static string, publicly visible (not ephemeral).
SLASH-HELP [V14] /help Ephemeral embed listing every slash command (deduplicated) and every dot-command with its description, usage, and alias list. Confirm long alias lists (e.g. .gif's 21 aliases) don't overflow/truncate the embed description past Discord's 4096-char limit — this is a real risk with EmbedBuilder validation in v14, which throws on overlength content instead of silently truncating the way some v13 paths tolerated.
SLASH-JOINT /joint Same phrase pool as .joint, posted publicly (not ephemeral), no roach-tracking interaction.
SLASH-STRAIN-1 /strain name:<partial of an existing strain> and use the autocomplete dropdown. Autocomplete suggests fuzzy matches (max 25) as you type; selecting one and submitting returns a full strain info embed (name, type, effects, flavor, rating, description).
SLASH-STRAIN-2 [V14] /strain name:<something with zero fuzzy matches> Autocomplete returns an empty list cleanly (no error) — confirms interaction.respond() with an empty array still works under v14.
SLASH-SETNICK-1 /setnick user:<a member the bot CAN rename> nickname:<short string> Ephemeral success message; nickname visibly changes on that member.
SLASH-SETNICK-2 /setnick user:<a member the bot CANNOT rename, e.g. a server owner or a role above the bot> Ephemeral error referencing lack of permission (error code 50013 path), not a generic crash.
SLASH-SETNICK-3 /setnick user:<any valid member> nickname:<a string over Discord's 32-char nickname limit> Ephemeral "nickname is too long" error (error code 50035 path).
SLASH-MEMBER-1 /member add newmember:<any user, including yourself testing as a non-owner if you have a second account> Reply confirms "Member Added" — note this currently has no permission gate, so test as a non-owner account too if feasible; if it succeeds for a non-owner, that's expected-but-concerning current behavior, not a regression (file separately from this test pass if you want it fixed).
SLASH-MEMBER-2 /member remove newmember:<a user previously added> Reply confirms "Member Removed."
SLASH-SETUP — expected-fail As the bot owner, run /setup against the dev DB (never run this against prod — it's destructive-by-intent even when working). Current code builds CREATE TABLE statements with single-quoted identifiers ('gifs' instead of a backtick or no quoting), which is invalid MySQL/MariaDB syntax. Expected: either the command reports success text while the tables silently fail to create (because errors are swallowed per-query without surfacing to the reply), or the DB driver throws. Confirm which, and confirm the existing dev DB tables are untouched either way (don't run this against tables with real content, since CREATE TABLE would also just fail harmlessly if they already exist — but verify no DROP/data loss occurs).
SLASH-RELOAD /reload Ephemeral "Reloaded!"; afterward, something added directly in the DB outside the bot (or via /save) since last startup shows up without a restart.
SLASH-CLOSEREQ-1 As owner: submit a .request first to get a known ID, then /closereq requestid:<that id>. Ephemeral confirmation; that request no longer appears in /view requests.
SLASH-CLOSEREQ-2 As a non-owner account (if available): /closereq requestid:<any id>. Ephemeral "You do not have permission to do that." — request status unchanged.
SLASH-VIEW-1 /view gifs, /view joints, /view pastas, /view requests — one at a time. Each returns an ephemeral paginated embed (10/page) with working ⬅️/➡️ buttons; see Section 5 for button-specific checks.
SLASH-SAVE-GIFURL /save gifurl url:<direct link to an image/gif> name:<new-test-name> Ephemeral confirmation "I've saved the GIF as <name>.gif"; immediately test <name>.gif as a dot-command and confirm it now resolves from the DB (not GIPHY).
SLASH-SAVE-JOINT /save joint joint-content:<new phrase> Ephemeral confirmation with joint emoji; run .joint enough times to confirm the new phrase can appear.
SLASH-SAVE-MD /save md advice-content:<new advice string> Ephemeral confirmation; run .md enough times to confirm it can appear.
SLASH-SAVE-PASTA /save pasta pasta-name:<new-test-name> pasta-content:<text> Ephemeral confirmation; <name>.pasta resolves the new content.
SLASH-SAVE-STRAIN /save strain name:<new> type:Indica effects:... flavor:... rating:... description:... Ephemeral confirmation; /strain name:<new> returns the saved data; autocomplete picks it up (may require /reload depending on timing — confirm either way).
SLASH-SAVE-GIFSEARCH — expected-fail /save gifsearch query:<anything> name:<anything> Hardcoded response: "Fuck off this is broken, come back later." No GIF is actually saved, no button/embed flow appears. This is intentionally disabled, not a bug to chase — confirm it fails the same way, not a new way.
SLASH-EDIT-GIF /edit gif name:<existing, via autocomplete> url:<new url> Ephemeral confirmation "I've updated <name>.gif"; dot-command <name>.gif now returns the new URL.
SLASH-EDIT-PASTA /edit pasta name:<existing, via autocomplete> content:<new text> Ephemeral confirmation; <name>.pasta returns new content.
SLASH-EDIT-STUBS — expected-fail If /edit ever exposes joint/md/strain subcommands in the picker (currently not declared in SlashCommandBuilder, so they shouldn't even appear) — confirm they're absent from the Discord UI subcommand picker. Only gif and pasta appear as selectable /edit subcommands.

5. Buttons & pagination [V14 critical]

Button/component interactions are one of the largest surface-area changes between v13 and v14 (MessageButton→ButtonBuilder, MessageActionRow→ActionRowBuilder, string style constants → ButtonStyle enum, customId access patterns on interaction.component).

ID Steps Expected
BTN-1 [V14] /view gifs with enough saved GIFs to span 2+ pages (add test entries via /save gifurl first if needed). Click ➡️. Embed updates in place (interaction.update, not a new message) to show page 2; footer page indicator increments.
BTN-2 [V14] From page 2 (or later), click ⬅️. Returns to the previous page correctly.
BTN-3 [V14] Navigate to the last page of /view gifs. ➡️ button is disabled (greyed out / unclickable) once on the last page.
BTN-4 [V14] On the first page, confirm ⬅️ is disabled. Button renders disabled, matches state === 'first' logic.
BTN-5 [V14] Repeat BTN-1 through BTN-4 for /view pastas, /view joints, /view requests. Same paging behavior independently for each content type.
BTN-6 Have a second Discord account (or ask someone) click the pagination buttons on an embed that YOU triggered via /view. Nothing happens — InteractionStorage gates button handling to the original invoking user's ID; the other user's click is silently ignored (no error shown to them either way, confirm it doesn't throw on the bot's end).
BTN-7 Let a /view embed's interaction sit idle for 5+ minutes, then click a pagination button. InteractionStorage entries expire after 300000ms (5 min) and are deleted; confirm whether clicking after expiry throws an unhandled error, silently fails, or gracefully re-initializes (ButtonHandlers.baseEvent does create a new InteractionStorage if one isn't found — confirm this path doesn't crash on a missing interaction.message.interaction reference, which is itself a nullable field).
BTN-8 [V14] — expected-fail /view requests (needs at least one open request), click "Close Requests" (if that button is visible — note: current requestsPageAR() builder only ever returns the prev/next row, the close button is constructed but never added to the returned ActionRowBuilder, so you may not even see it in the UI; if you don't see it, note that as the current actual behavior and skip the modal test below). If you DO see a "Close Requests" button, click it. Expected per current code: interaction.showModal() is called with a Modal class that Embeds.js never imports — this should throw a ReferenceError: Modal is not defined server-side, surfacing to you as "This interaction failed" in Discord with no modal appearing. Confirm it fails this way (or document if v14's ModalBuilder happens to already be globally available some other way and it actually works — that would be a notable surprise).
BTN-9 /save gifsearch (confirm this is still the disabled stub per SLASH-SAVE-GIFSEARCH) — the GIF-search button row (prev/confirm/next/cancel) should never actually render since the subcommand bails out immediately. No buttons appear; dead code in ButtonHandlers.gifSearchPage/actionRows.gifSearchAR remains unreachable. Flag only if this somehow becomes reachable post-migration.

6. Permission-gated features

ID Steps Expected
PERM-1 As the bot owner, run /closereq and /setup. Both execute their owner-gated logic (see Section 4 for specific expectations, including /setup's expected-fail).
PERM-2 As a non-owner, run /closereq requestid:<any>. "You do not have permission to do that." — no DB change.
PERM-3 As a non-owner, attempt /setup. "Sorry, you don't have permission to do that." — no tables created/attempted.
PERM-4 As a non-owner, run /setnick targeting a member above the bot's role, or targeting yourself with the bot lacking Manage Nicknames server permission entirely. Discord's own permission system blocks it at the API level before NodBot's own (nonexistent) check would matter — confirm the 50013 error path (SLASH-SETNICK-2) still triggers correctly, since this is NodBot's only real permission-error handling path.

7. External-dependency resilience

These exercise what happens when NodBot's three outbound APIs (GIPHY, aviationweather.gov, datis.clowd.io) are slow, empty, or erroring — relevant because v14 changes how some async/promise rejection paths surface uncaught errors (stricter unhandled rejection behavior in newer Node + discord.js combos).

ID Steps Expected
EXT-1 Send .gif with a deliberately nonsense query unlikely to match anything on GIPHY (e.g. a long random string). "Sorry I was unable to find a GIF of ..." reply, no crash, no hung interaction.
EXT-2 Send .metar with a syntactically valid-looking but nonexistent ICAO not in the cached list. Caught Invalid ICAO ID Detected error reply (from parseICAOs's validation against config.icaoIds), not a raw API error leaking to the user.
EXT-3 Send .datis for an ICAO present in the D-ATIS station list but currently has no active ATIS broadcast (common for small airports overnight). "No D-ATIS available for the specified ICAO ID." — not a crash from the underlying API's error shape.

8. Known-broken / dead features (expected-fail baseline)

Included so a regression run distinguishes "still broken the same way" from "broken differently" or "unexpectedly started working" — any of those three outcomes is useful signal, especially right after a major dependency bump.

ID Steps Expected (baseline, pre-migration)
DEAD-1 /save gifsearch Hardcoded "Fuck off this is broken, come back later." reply. See SLASH-SAVE-GIFSEARCH.
DEAD-2 Attempt to reach /edit joint, /edit md, /edit strain via the Discord slash command picker. These subcommands don't exist in SlashCommandBuilder — they're not selectable at all, not even a stub reply.
DEAD-3 closeRequests button, if reachable. ReferenceError: Modal is not defined, surfaced to you as a failed interaction. See BTN-8.
DEAD-4 /setup, run against dev DB only. Invalid SQL identifier quoting; see SLASH-SETUP.
DEAD-5 Confirm no command, button, or automatic flow ever invokes fn.checkMembership, the paywall embeds (noMembership, membershipAnnouncement), or the DALL-E/GPT-3.5/upload.openai code paths. None of these are reachable from any current command — this is intentional dead code, not a bug. No test action needed beyond confirming /member add still doesn't gate anything (see SLASH-MEMBER-1).

Appendix A: Known fragile areas (prioritize these if time is short)

Per prior experience with this codebase, weight extra attention toward:

  • Embed/component rendering — every MessageEmbed/MessageButton/MessageActionRow construction site across functions.js, CustomModules/Embeds.js, and CustomModules/ButtonHandlers.js. String-based style/color constants ('BLUE', 'PRIMARY', 'SECONDARY', 'DANGER') are the most likely silent-breakage point if the v14 migration is incomplete in some files but not others.
  • Startup/intents/partials — if the bot fails to log in or ready never fires, nothing else in this suite is reachable. Always run Section 1 and Section 2 first and treat a failure there as a hard stop.

Appendix B: v13→v14 API change map

Reference for tagging/triaging failures found while running [V14] tests — maps the user-visible symptom to the underlying v13→v14 code change likely responsible, based on this codebase's actual usage.

Area v13 (current code) v14 Where it's used here
Client intents Intents string array ('GUILDS', etc.) GatewayIntentBits enum main.js, _list-guilds.js
Client partials 'CHANNEL', 'MESSAGE' strings Partials enum main.js
Embeds new Discord.MessageEmbed() EmbedBuilder functions.js, CustomModules/Embeds.js
Buttons new MessageButton() ButtonBuilder CustomModules/ButtonHandlers.js, CustomModules/Embeds.js
Action rows new MessageActionRow() ActionRowBuilder same as above
Text inputs / modals TextInputComponent, Modal (currently unimported/broken) TextInputBuilder, ModalBuilder CustomModules/Embeds.js
Button styles 'PRIMARY'/'SECONDARY'/'DANGER' strings ButtonStyle.Primary etc. enum CustomModules/Embeds.js
Colors 'BLUE' string Colors.Blue or numeric functions.js (embeds.help)
Interaction type checks interaction.isCommand() interaction.isChatInputCommand() (isCommand() is deprecated/removed) main.js
Slash command builder @discordjs/builders SlashCommandBuilder (0.16, pre-v14) bundled into discord.js v14 directly every slash-commands/*.js file
REST/API versioning @discordjs/rest canary + discord-api-types v9 (Routes from /v9) stable @discordjs/rest + discord-api-types v10 _deploy-commands.js, _deploy-global.js, _clear-commands.js
Permission flags not directly used in this codebase (relies on Discord API error codes 50013/50035 instead) PermissionFlagsBits enum if ever added slash-commands/setnick.js (indirectly)
Message Content implicit via grandfather exemption, no explicit intent same exemption still applies, but intent should be declared explicitly for clarity/future-proofing all dot-commands
Attachments not used directly (GIFs are sent as URLs, not file uploads) n/a —
fetchReply/reply-then-edit patterns interaction.reply()/interaction.deferReply()/interaction.editReply() used throughout, API-compatible across versions same all slash commands