sdp-request-watcher/docs/NOTES.md

Notes

SDP v3 API gotchas

Docs: https://www.manageengine.com/products/service-desk/sdpod-v3-api/getting-started/search-criteria.html

• Condition names have spaces: greater than, is not, lesser than. greater_than isn't valid, and SDP silently ignored the entire search criteria because of it, returning every ticket (124,050). If a query's total_count is huge, suspect the criteria first.
• Paging: row_count is at most 100. Use start_index, and stop when list_info.has_more_rows is false. Ask for get_total_count: true on the first page. getAllRequests dedupes by ID and stops at 20 pages or when the total is unreachable.
• Multiple values: use a values: [...] array, and join criteria with logical_operator: 'AND' (see allOf() in tickets.js).
• "Technician is empty": written as { field: 'technician', condition: 'is' } with no value. This isn't documented, since the docs only show the "is not" form, but it's verified to work.
• Field names that work in filters: site.name, group.name, technician.name, status.name, last_updated_time.
• Group names: the ManageEngine UI shows groups as "Customer Support « KVCC", but the API name is plain "Customer Support".
• Sign-in scope: SDPOnDemand.requests.READ is enough for every SDP query so far. AaaServer.profile.READ is added only for the signed-in user's name (GET accounts.zoho.com/oauth/user/info). If that call fails, sign-in still works. Changing the scopes means everyone signs in again.

Zoho OAuth gotchas

• Client type: web sign-in needs a Server-based Application client. The old Non-Browser (device-code) client can't do redirects. The redirect URI must match PUBLIC_URL/auth/callback exactly.
• Refresh tokens: you only get one with access_type=offline, and prompt=consent makes Zoho issue one on every sign-in. Access tokens last about an hour; renew with POST accounts.zoho.com/oauth/v2/token, grant_type=refresh_token.
• 20 refresh tokens per user per client: Zoho silently deletes the oldest when a 21st is issued. Each browser sign-in creates one, which is why Sign out revokes (POST /oauth/v2/token/revoke?token=...). A user signed in on more than 20 browsers would find the oldest one signed out.
• Errors come back as HTTP 200: a bad code or refresh token returns { "error": "invalid_code" } with status 200. auth.js checks for access_token instead of relying on axios throwing.
• Data centre: everything points at accounts.zoho.com (US). The callback also gets location and accounts-server parameters, which would matter only for users in another Zoho DC.

Deployment gotchas

• NPM runs in Docker, so localhost in a proxy host is NPM's own container. For a bare-metal test run, forward to the gateway of NPM's Docker network and allow that subnet through ufw. The production container is reached by name (sdp-dashboard:3000) because it joins NPM's network.
• Port 3000 is Gitea's on the VPS. Bare-metal test runs use PORT=3005. Never put PORT in the production .env; the container listens on 3000, and its health check and the NPM host expect that.
• Cloudflare proxy and the NPM access list: a proxied record can make NPM see Cloudflare's IPs, so the allow list blocks everyone. The fix is DNS only, or real_ip_header CF-Connecting-IP; in the proxy host's Advanced config. The live test on 2026-10-15 worked with the access list (an MCCS-listed IP got in, mobile data got a 403).

Decisions

• 2026-10-15: MIT license. Copyright Skylar Grant.
• 2026-10-15: OpenWeatherMap for weather. The free tier covers current weather and the 5-day forecast, and results are cached for 10 minutes. The National Weather Service API was the alternative (free, no key), but it needs three calls per lookup.
• 2026-10-15: Unassigned counter uses its own status exclusions (UNASSIGNED_EXCLUDED_STATUSES) to match the ManageEngine view. The debug list is kept but hidden (SHOW_UNASSIGNED_LIST in index.html).
• 2026-10-15: Per-user Zoho sign-in replaces the shared service token. This reverses the earlier "one service token plus region config" decision, and drops the ADMIN_KEY and data/token.json from phase 1. Leadership's concern was that anyone who reached the page could see data while a shared token was live. Now every request needs the viewer's own sign-in, and NPM limits the site to MCCS addresses. The cost is that each open dashboard polls SDP itself; acceptable for a small team.
• 2026-10-15: Sessions in an encrypted cookie, not a server store. AES-256-GCM with a key from SESSION_SECRET, HttpOnly, SameSite=Lax (Lax so the cookie survives the redirect back from Zoho), and Secure when PUBLIC_URL is https. The container needs no volume or database, and restarts don't sign anyone out. Renewing the access token rewrites the cookie but keeps startedAt, so SESSION_DAYS is a hard limit. Concurrent calls with an expired token share one renewal (renewals map in session.js).
• 2026-10-15: The browser runs the checks and keeps the closed feed (first load asks for 12 hours, then 10-minute windows every minute). The server has no per-user timers or state. This also suits per-browser settings later: the browser will send its settings with each call.
• 2026-10-15: Per-user settings are stored on the server, keyed by Zoho email (lowercased), in DATA_DIR/settings.json. Chosen over the planned localStorage so settings follow people between browsers and no export/import is needed. This gives the container state, so compose mounts the sdp-data volume at /app/data. Only values that differ from .env are stored, so .env changes still reach people who never changed that field. A plain JSON file, rewritten via temp file and rename, is enough for a small team with one process; move to SQLite if that stops being true.
• 2026-10-15: The server reads settings itself rather than the browser sending them with each call, as first planned. The browser can only change them through PUT /api/settings, which validates everything (LIMITS in settings.js). That's for protecting SDP and the server, not access control: queries use the user's own token, and everyone at MCCS can see every site in SDP anyway.
• 2026-10-15: Name suggestions without a new scope. /api/names runs one unscoped query (last 30 days, up to 500 tickets, names only) with the caller's token, at most hourly for everyone, and merges into DATA_DIR/names.json so the list grows over time. Typed names are still accepted; a name matching a suggestion except for case is saved with SDP's spelling, because the technician filters compare exactly. Proper pick lists from SDP's setup/users APIs would need another scope and a re-sign-in for everyone.
• 2026-10-15: Weather locations are a fixed list (src/places.js), not free zip codes, so a browser can't make the server look up arbitrary places and the per-zip cache stays small. The .env zip is added to the list if it's missing.
• 2026-10-15: Status rules stay in .env (STATUS_NAMES, UNASSIGNED_EXCLUDED_STATUSES). They're the help desk's definition of open and closed, not a personal preference. Hiding a status column is per user, but hidden columns still count toward Total open.
• 2026-10-15: Sound stays per browser (localStorage), not in the saved settings, so a muted desk browser doesn't mute the wallboard signed in as the same person.
• 2026-10-15: No published ports in compose. Docker's port publishing bypasses ufw, which would skip NPM's IP access list. The container joins NPM's network instead.
• 2026-10-15: CI uses plain docker commands on the self-hosted runner, not the buildx or login actions, so there are fewer moving parts on the runner. Pushes both the tag and latest.

Known issues

• The status table scrolls sideways once there are more statuses than fit, because the page is capped at 1,320px. Consider widening the page or narrowing the sidebar.
• All statuses not in STATUS_NAMES count as open, including "Need Information" and "On Hold". This is intentional; people can hide those columns in Settings.
• If Zoho doesn't return an email at sign-in (the profile call failed), that session can't save settings and uses the defaults; signing in again fixes it.
• No screenshots: images/ was removed on 2026-10-15 and new ones are due.