Dataforth UI push + dedup + refactor, GuruRMM roadmap evolution, Azure signing setup

Dataforth (projects/dataforth-dos/):
- UI feature: row coloring + PUSH/RE-PUSH buttons + Website Status filter
- Database dedup to one row per SN (2.89M -> 469K rows, UNIQUE constraint added)
- Import logic handles FAIL -> PASS retest transition
- Refactored upload-to-api.js to render datasheets in-memory (dropped For_Web filesystem dep)
- Bulk pushed 170,984 records to Hoffman API
- Statistical sanity check: 100/100 stamped SNs verified on Hoffman

GuruRMM (projects/msp-tools/guru-rmm/):
- ROADMAP.md: added Terminology (5-tier hierarchy), Tunnel Channels Phase 2,
  Logging/Audit/Observability, Multi-tenancy, Modular Architecture,
  Protocol Versioning, Certificates sections + Decisions Log
- CONTEXT.md: hierarchy table, new anti-patterns (bootstrap sacred,
  no cross-module imports), revised next-steps priorities

Session logs for both projects.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

.claude/scheduled_tasks.lock

@@ -0,0 +1 @@
+{"sessionId":"cfd47b51-999e-4ed8-ba99-034493914f76","pid":30732,"acquiredAt":1776026819140}

.claude/skills/skill-creator/SKILL.md

@@ -0,0 +1,117 @@
+---
+name: skill-creator
+description: |
+  Create new Claude Code custom skills and slash commands. Use when the user wants to create a new skill,
+  add a slash command, build a custom command, or set up a new automation. Guides through the process of
+  defining the skill's purpose, triggers, and implementation, then generates the proper file structure.
+---
+
+# Skill Creator
+
+You help the user create new Claude Code custom skills and slash commands.
+
+## Two Types of Custom Extensions
+
+### 1. Skills (`.claude/skills/{name}/SKILL.md`)
+- Rich, multi-purpose capabilities with automatic invocation triggers
+- Can include supporting files (scripts, references, checklists)
+- Best for: complex behaviors, design patterns, validation workflows, integrations
+
+### 2. Slash Commands (`.claude/commands/{name}.md`)
+- Simple, user-invoked commands triggered by `/{name}`
+- Single markdown file with instructions
+- Best for: workflows the user explicitly triggers, task automation, shortcuts
+- Can accept arguments via `$ARGUMENTS`
+
+## Creation Process
+
+### Step 1: Gather Requirements
+
+Ask the user:
+1. **What should this skill/command do?** (core purpose)
+2. **Skill or command?** Help them decide:
+   - If it should run automatically in response to certain actions -> **Skill**
+   - If the user will invoke it explicitly with `/{name}` -> **Command**
+   - If unsure, recommend based on the use case
+3. **Name** - short, kebab-case identifier (e.g., `code-review`, `deploy-check`)
+4. **When should it trigger?** (for skills: automatic triggers; for commands: typical usage)
+
+### Step 2: Generate the Files
+
+#### For Skills
+
+Create `.claude/skills/{name}/SKILL.md`:
+
+```markdown
+---
+name: {name}
+description: |
+  {Detailed description. This is used for discovery/matching, so be specific about
+  when this skill should be invoked. Include trigger keywords and example scenarios.}
+---
+
+# {Skill Title}
+
+{Clear instructions for what Claude should do when this skill is invoked.}
+
+## When to Invoke
+
+{List specific triggers - file types, actions, keywords that should activate this skill.}
+
+## Workflow
+
+{Step-by-step process the skill follows.}
+
+## Guidelines
+
+{Rules, patterns, and best practices to follow.}
+```
+
+#### For Commands
+
+Create `.claude/commands/{name}.md`:
+
+```markdown
+---
+description: {One-line description shown in command list}
+---
+
+# {Command Title}
+
+{Instructions for what Claude should do when the user runs /{name}.}
+
+## Arguments
+
+If the command accepts arguments, reference them via `$ARGUMENTS`.
+
+## Workflow
+
+{Step-by-step process.}
+```
+
+### Step 3: Register and Validate
+
+After creating the files:
+1. Confirm the file was created in the correct location
+2. Tell the user they can invoke it:
+   - Skills: Explain the automatic triggers or manual invocation via `/skill-name`
+   - Commands: Tell them to use `/{name}` or `/{name} arguments`
+3. Remind them to update CLAUDE.md's Commands & Skills table if they want it documented there
+
+## Quality Checklist
+
+Before finalizing, verify:
+- [ ] Description is detailed enough for Claude to match it to relevant situations
+- [ ] Instructions are clear and actionable (Claude will follow them literally)
+- [ ] The skill/command doesn't duplicate an existing one
+- [ ] File is in the correct location (`.claude/skills/` or `.claude/commands/`)
+- [ ] Name uses kebab-case and is concise
+- [ ] For skills with auto-triggers: triggers are specific enough to avoid false positives
+
+## Tips for Good Skills/Commands
+
+- **Be specific in descriptions** - vague descriptions lead to missed or false invocations
+- **Include examples** in the instructions so Claude understands edge cases
+- **Keep scope focused** - one skill per concern, don't create mega-skills
+- **Test after creation** - have the user try invoking it to verify behavior
+- **Reference existing patterns** - look at `.claude/skills/` and `.claude/commands/` for examples

.claude/skills/stop-slop/SKILL.md

@@ -0,0 +1,105 @@
+---
+name: stop-slop
+description: |
+  Enforce high-quality, slop-free output in all Claude responses. MANDATORY AUTOMATIC INVOCATION:
+  This skill is always active. It governs how Claude writes text, code comments, commit messages,
+  documentation, and any other output. Detects and eliminates generic AI filler, hollow phrases,
+  unnecessary verbosity, and performative enthusiasm. Applies to all output — conversation, code,
+  docs, and generated content.
+---
+
+# Stop Slop
+
+You are a direct, competent engineer. Write like one. Every word must earn its place.
+
+## Always-On Rules
+
+These rules apply to ALL output at ALL times. No exceptions.
+
+### Banned Patterns -- Never Write These
+
+**Performative enthusiasm and filler openers:**
+- "Great question!", "Excellent point!", "That's a really interesting..."
+- "Certainly!", "Absolutely!", "Of course!", "Sure thing!"
+- "I'd be happy to help!", "Let me help you with that!"
+- "Good news!", "Here's the exciting part..."
+
+**Hollow transitions and hedging:**
+- "It's worth noting that..." (just state it)
+- "It's important to remember..." (just state it)
+- "As you can see..." / "As we discussed..."
+- "Basically..." / "Essentially..." / "Fundamentally..."
+- "In order to..." (use "to")
+- "It should be noted that..." (just note it)
+- "At the end of the day..."
+- "Moving forward..."
+
+**Unnecessary meta-commentary:**
+- "Let me explain..." (just explain)
+- "I'll now..." / "Next, I'll..." (just do it)
+- "Here's what I found..." (just show it)
+- "Let me break this down..." (just break it down)
+
+**Trailing summaries and sign-offs:**
+- Restating what was just done at the end of a response
+- "Let me know if you have any questions!"
+- "Hope this helps!"
+- "Feel free to ask if you need anything else!"
+- "Happy coding!" / "Happy hacking!"
+- Any variation of "don't hesitate to reach out"
+
+**Weasel words and padding:**
+- "Very", "really", "quite", "rather", "somewhat", "fairly"
+- "Just" (when used as filler, not as "only")
+- "Simply" (when the thing isn't simple, or as filler)
+- "Actually" (at start of sentences, as filler)
+- "Obviously" / "Clearly" (if it were obvious, you wouldn't say it)
+
+**Sycophantic agreement:**
+- "You're absolutely right that..."
+- "That's a great approach!"
+- "What a thoughtful question!"
+- Praising the user's code/ideas before giving feedback
+
+### Writing Standards
+
+**Lead with the answer.** Don't build up to it. State the conclusion, then support it if needed.
+
+**One sentence beats three.** If you can say it shorter, do. Compress ruthlessly.
+
+**No preamble.** Start with the substance. Drop throat-clearing intros.
+
+**No postamble.** End when the content ends. Don't summarize what you just said. Don't offer further help.
+
+**Be specific.** "This fails because X" not "There might be some issues with this approach."
+
+**Code comments: only when non-obvious.** Don't add comments that restate what the code does. Comment the *why*, not the *what*. Most code needs zero comments.
+
+**Commit messages: state the change.** Not "This commit updates the..." -- just "Update X to handle Y."
+
+**Error messages: state what went wrong and what to do.** Not "Oops! It looks like something went wrong."
+
+### Calibration Examples
+
+**Slop:**
+> Great question! Let me help you with that. So basically, what's happening here is that the function is essentially trying to parse the input string. It's worth noting that this can sometimes fail if the input isn't valid JSON. I'd recommend wrapping it in a try-catch block to handle any potential errors that might occur. Let me know if you have any questions!
+
+**Clean:**
+> The function fails on invalid JSON. Wrap it in try-catch:
+> ```js
+> try { return JSON.parse(input); } catch { return null; }
+> ```
+
+**Slop:**
+> I've successfully updated the configuration file to include the new database connection settings. The changes include adding the host, port, username, and password fields as requested. Everything should be working correctly now. Feel free to test it out and let me know if you run into any issues!
+
+**Clean:**
+> Updated the database config with the new connection settings.
+
+### What This Skill Does NOT Do
+
+- It does not make responses terse to the point of being unhelpful
+- It does not remove necessary technical explanation
+- It does not prevent friendly, human tone -- just fake enthusiasm
+- It does not restrict response length when length is warranted by complexity
+- Thoroughness is good. Fluff is not. Know the difference.

.claude/skills/theme-factory/LICENSE.txt

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

.claude/skills/theme-factory/SKILL.md

@@ -0,0 +1,59 @@
+---
+name: theme-factory
+description: Toolkit for styling artifacts with a theme. These artifacts can be slides, docs, reportings, HTML landing pages, etc. There are 10 pre-set themes with colors/fonts that you can apply to any artifact that has been creating, or can generate a new theme on-the-fly.
+license: Complete terms in LICENSE.txt
+---
+
+
+# Theme Factory Skill
+
+This skill provides a curated collection of professional font and color themes themes, each with carefully selected color palettes and font pairings. Once a theme is chosen, it can be applied to any artifact.
+
+## Purpose
+
+To apply consistent, professional styling to presentation slide decks, use this skill. Each theme includes:
+- A cohesive color palette with hex codes
+- Complementary font pairings for headers and body text
+- A distinct visual identity suitable for different contexts and audiences
+
+## Usage Instructions
+
+To apply styling to a slide deck or other artifact:
+
+1. **Show the theme showcase**: Display the `theme-showcase.pdf` file to allow users to see all available themes visually. Do not make any modifications to it; simply show the file for viewing.
+2. **Ask for their choice**: Ask which theme to apply to the deck
+3. **Wait for selection**: Get explicit confirmation about the chosen theme
+4. **Apply the theme**: Once a theme has been chosen, apply the selected theme's colors and fonts to the deck/artifact
+
+## Themes Available
+
+The following 10 themes are available, each showcased in `theme-showcase.pdf`:
+
+1. **Ocean Depths** - Professional and calming maritime theme
+2. **Sunset Boulevard** - Warm and vibrant sunset colors
+3. **Forest Canopy** - Natural and grounded earth tones
+4. **Modern Minimalist** - Clean and contemporary grayscale
+5. **Golden Hour** - Rich and warm autumnal palette
+6. **Arctic Frost** - Cool and crisp winter-inspired theme
+7. **Desert Rose** - Soft and sophisticated dusty tones
+8. **Tech Innovation** - Bold and modern tech aesthetic
+9. **Botanical Garden** - Fresh and organic garden colors
+10. **Midnight Galaxy** - Dramatic and cosmic deep tones
+
+## Theme Details
+
+Each theme is defined in the `themes/` directory with complete specifications including:
+- Cohesive color palette with hex codes
+- Complementary font pairings for headers and body text
+- Distinct visual identity suitable for different contexts and audiences
+
+## Application Process
+
+After a preferred theme is selected:
+1. Read the corresponding theme file from the `themes/` directory
+2. Apply the specified colors and fonts consistently throughout the deck
+3. Ensure proper contrast and readability
+4. Maintain the theme's visual identity across all slides
+
+## Create your Own Theme
+To handle cases where none of the existing themes work for an artifact, create a custom theme. Based on provided inputs, generate a new theme similar to the ones above. Give the theme a similar name describing what the font/color combinations represent. Use any basic description provided to choose appropriate colors/fonts. After generating the theme, show it for review and verification. Following that, apply the theme as described above.

.claude/skills/theme-factory/themes/arctic-frost.md

@@ -0,0 +1,19 @@
+# Arctic Frost
+
+A cool and crisp winter-inspired theme that conveys clarity, precision, and professionalism.
+
+## Color Palette
+
+- **Ice Blue**: `#d4e4f7` - Light backgrounds and highlights
+- **Steel Blue**: `#4a6fa5` - Primary accent color
+- **Silver**: `#c0c0c0` - Metallic accent elements
+- **Crisp White**: `#fafafa` - Clean backgrounds and text
+
+## Typography
+
+- **Headers**: DejaVu Sans Bold
+- **Body Text**: DejaVu Sans
+
+## Best Used For
+
+Healthcare presentations, technology solutions, winter sports, clean tech, pharmaceutical content.

.claude/skills/theme-factory/themes/botanical-garden.md

@@ -0,0 +1,19 @@
+# Botanical Garden
+
+A fresh and organic theme featuring vibrant garden-inspired colors for lively presentations.
+
+## Color Palette
+
+- **Fern Green**: `#4a7c59` - Rich natural green
+- **Marigold**: `#f9a620` - Bright floral accent
+- **Terracotta**: `#b7472a` - Earthy warm tone
+- **Cream**: `#f5f3ed` - Soft neutral backgrounds
+
+## Typography
+
+- **Headers**: DejaVu Serif Bold
+- **Body Text**: DejaVu Sans
+
+## Best Used For
+
+Garden centers, food presentations, farm-to-table content, botanical brands, natural products.

.claude/skills/theme-factory/themes/desert-rose.md

@@ -0,0 +1,19 @@
+# Desert Rose
+
+A soft and sophisticated theme with dusty, muted tones perfect for elegant presentations.
+
+## Color Palette
+
+- **Dusty Rose**: `#d4a5a5` - Soft primary color
+- **Clay**: `#b87d6d` - Earthy accent
+- **Sand**: `#e8d5c4` - Warm neutral backgrounds
+- **Deep Burgundy**: `#5d2e46` - Rich dark contrast
+
+## Typography
+
+- **Headers**: FreeSans Bold
+- **Body Text**: FreeSans
+
+## Best Used For
+
+Fashion presentations, beauty brands, wedding planning, interior design, boutique businesses.

.claude/skills/theme-factory/themes/forest-canopy.md

@@ -0,0 +1,19 @@
+# Forest Canopy
+
+A natural and grounded theme featuring earth tones inspired by dense forest environments.
+
+## Color Palette
+
+- **Forest Green**: `#2d4a2b` - Primary dark green
+- **Sage**: `#7d8471` - Muted green accent
+- **Olive**: `#a4ac86` - Light accent color
+- **Ivory**: `#faf9f6` - Backgrounds and text
+
+## Typography
+
+- **Headers**: FreeSerif Bold
+- **Body Text**: FreeSans
+
+## Best Used For
+
+Environmental presentations, sustainability reports, outdoor brands, wellness content, organic products.

.claude/skills/theme-factory/themes/golden-hour.md

@@ -0,0 +1,19 @@
+# Golden Hour
+
+A rich and warm autumnal palette that creates an inviting and sophisticated atmosphere.
+
+## Color Palette
+
+- **Mustard Yellow**: `#f4a900` - Bold primary accent
+- **Terracotta**: `#c1666b` - Warm secondary color
+- **Warm Beige**: `#d4b896` - Neutral backgrounds
+- **Chocolate Brown**: `#4a403a` - Dark text and anchors
+
+## Typography
+
+- **Headers**: FreeSans Bold
+- **Body Text**: FreeSans
+
+## Best Used For
+
+Restaurant presentations, hospitality brands, fall campaigns, cozy lifestyle content, artisan products.

.claude/skills/theme-factory/themes/midnight-galaxy.md

@@ -0,0 +1,19 @@
+# Midnight Galaxy
+
+A dramatic and cosmic theme with deep purples and mystical tones for impactful presentations.
+
+## Color Palette
+
+- **Deep Purple**: `#2b1e3e` - Rich dark base
+- **Cosmic Blue**: `#4a4e8f` - Mystical mid-tone
+- **Lavender**: `#a490c2` - Soft accent color
+- **Silver**: `#e6e6fa` - Light highlights and text
+
+## Typography
+
+- **Headers**: FreeSans Bold
+- **Body Text**: FreeSans
+
+## Best Used For
+
+Entertainment industry, gaming presentations, nightlife venues, luxury brands, creative agencies.

.claude/skills/theme-factory/themes/modern-minimalist.md

@@ -0,0 +1,19 @@
+# Modern Minimalist
+
+A clean and contemporary theme with a sophisticated grayscale palette for maximum versatility.
+
+## Color Palette
+
+- **Charcoal**: `#36454f` - Primary dark color
+- **Slate Gray**: `#708090` - Medium gray for accents
+- **Light Gray**: `#d3d3d3` - Backgrounds and dividers
+- **White**: `#ffffff` - Text and clean backgrounds
+
+## Typography
+
+- **Headers**: DejaVu Sans Bold
+- **Body Text**: DejaVu Sans
+
+## Best Used For
+
+Tech presentations, architecture portfolios, design showcases, modern business proposals, data visualization.

.claude/skills/theme-factory/themes/ocean-depths.md

@@ -0,0 +1,19 @@
+# Ocean Depths
+
+A professional and calming maritime theme that evokes the serenity of deep ocean waters.
+
+## Color Palette
+
+- **Deep Navy**: `#1a2332` - Primary background color
+- **Teal**: `#2d8b8b` - Accent color for highlights and emphasis
+- **Seafoam**: `#a8dadc` - Secondary accent for lighter elements
+- **Cream**: `#f1faee` - Text and light backgrounds
+
+## Typography
+
+- **Headers**: DejaVu Sans Bold
+- **Body Text**: DejaVu Sans
+
+## Best Used For
+
+Corporate presentations, financial reports, professional consulting decks, trust-building content.

.claude/skills/theme-factory/themes/sunset-boulevard.md

@@ -0,0 +1,19 @@
+# Sunset Boulevard
+
+A warm and vibrant theme inspired by golden hour sunsets, perfect for energetic and creative presentations.
+
+## Color Palette
+
+- **Burnt Orange**: `#e76f51` - Primary accent color
+- **Coral**: `#f4a261` - Secondary warm accent
+- **Warm Sand**: `#e9c46a` - Highlighting and backgrounds
+- **Deep Purple**: `#264653` - Dark contrast and text
+
+## Typography
+
+- **Headers**: DejaVu Serif Bold
+- **Body Text**: DejaVu Sans
+
+## Best Used For
+
+Creative pitches, marketing presentations, lifestyle brands, event promotions, inspirational content.

.claude/skills/theme-factory/themes/tech-innovation.md

@@ -0,0 +1,19 @@
+# Tech Innovation
+
+A bold and modern theme with high-contrast colors perfect for cutting-edge technology presentations.
+
+## Color Palette
+
+- **Electric Blue**: `#0066ff` - Vibrant primary accent
+- **Neon Cyan**: `#00ffff` - Bright highlight color
+- **Dark Gray**: `#1e1e1e` - Deep backgrounds
+- **White**: `#ffffff` - Clean text and contrast
+
+## Typography
+
+- **Headers**: DejaVu Sans Bold
+- **Body Text**: DejaVu Sans
+
+## Best Used For
+
+Tech startups, software launches, innovation showcases, AI/ML presentations, digital transformation content.

projects/dataforth-dos/datasheet-pipeline/current-testdatadb/db.js

@@ -0,0 +1,139 @@
+/**
+ * PostgreSQL Database Abstraction Layer
+ *
+ * Provides a connection pool and helper methods for the TestDataDB app.
+ * Replaces better-sqlite3 singleton with pg.Pool.
+ *
+ * Environment variables (all optional, defaults connect to local PG):
+ *   PGHOST (default: localhost)
+ *   PGPORT (default: 5432)
+ *   PGUSER (default: testdatadb_app)
+ *   PGPASSWORD (default: DfTestDB2026!)
+ *   PGDATABASE (default: testdatadb)
+ */
+
+const { Pool } = require('pg');
+
+const pool = new Pool({
+    host: process.env.PGHOST || 'localhost',
+    port: parseInt(process.env.PGPORT || '5432', 10),
+    user: process.env.PGUSER || 'testdatadb_app',
+    password: process.env.PGPASSWORD || 'DfTestDB2026!',
+    database: process.env.PGDATABASE || 'testdatadb',
+    max: 20,
+    idleTimeoutMillis: 30000,
+    connectionTimeoutMillis: 5000,
+});
+
+pool.on('error', (err) => {
+    console.error(`[${new Date().toISOString()}] [PG POOL ERROR] ${err.message}`);
+});
+
+/**
+ * Convert SQLite-style ? placeholders to PostgreSQL $1, $2, ... placeholders.
+ * Skips ? inside single-quoted strings.
+ */
+function convertPlaceholders(sql) {
+    let idx = 0;
+    let inString = false;
+    let result = '';
+    for (let i = 0; i < sql.length; i++) {
+        const ch = sql[i];
+        if (ch === "'" && (i === 0 || sql[i - 1] !== '\\')) {
+            inString = !inString;
+            result += ch;
+        } else if (ch === '?' && !inString) {
+            idx++;
+            result += '$' + idx;
+        } else {
+            result += ch;
+        }
+    }
+    return result;
+}
+
+/**
+ * Execute a query, return all rows.
+ * @param {string} sql - SQL with ? or $N placeholders
+ * @param {Array} params - Parameter values
+ * @returns {Promise<Array>} rows
+ */
+async function query(sql, params = []) {
+    const pgSql = sql.includes('?') ? convertPlaceholders(sql) : sql;
+    const result = await pool.query(pgSql, params);
+    return result.rows;
+}
+
+/**
+ * Execute a query, return the first row or null.
+ */
+async function queryOne(sql, params = []) {
+    const rows = await query(sql, params);
+    return rows[0] || null;
+}
+
+/**
+ * Execute a statement (INSERT/UPDATE/DELETE), return { rowCount }.
+ */
+async function execute(sql, params = []) {
+    const pgSql = sql.includes('?') ? convertPlaceholders(sql) : sql;
+    const result = await pool.query(pgSql, params);
+    return { rowCount: result.rowCount, rows: result.rows };
+}
+
+/**
+ * Run a function inside a transaction.
+ * The callback receives a client with query/execute helpers.
+ * @param {Function} fn - async (client) => result
+ * @returns {Promise<*>} result of fn
+ */
+async function transaction(fn) {
+    const client = await pool.connect();
+    try {
+        await client.query('BEGIN');
+
+        const txClient = {
+            async query(sql, params = []) {
+                const pgSql = sql.includes('?') ? convertPlaceholders(sql) : sql;
+                const result = await client.query(pgSql, params);
+                return result.rows;
+            },
+            async queryOne(sql, params = []) {
+                const rows = await txClient.query(sql, params);
+                return rows[0] || null;
+            },
+            async execute(sql, params = []) {
+                const pgSql = sql.includes('?') ? convertPlaceholders(sql) : sql;
+                const result = await client.query(pgSql, params);
+                return { rowCount: result.rowCount, rows: result.rows };
+            },
+            // Direct pg client access for COPY or other advanced operations
+            raw: client,
+        };
+
+        const result = await fn(txClient);
+        await client.query('COMMIT');
+        return result;
+    } catch (err) {
+        await client.query('ROLLBACK');
+        throw err;
+    } finally {
+        client.release();
+    }
+}
+
+/**
+ * Close the pool (for graceful shutdown).
+ */
+async function close() {
+    await pool.end();
+}
+
+/**
+ * Get the raw pool (for advanced use like COPY).
+ */
+function getPool() {
+    return pool;
+}
+
+module.exports = { query, queryOne, execute, transaction, close, getPool, convertPlaceholders };

projects/dataforth-dos/datasheet-pipeline/current-testdatadb/export-datasheets.js

@@ -0,0 +1,257 @@
+/**
+ * Export Datasheets
+ *
+ * Generates TXT datasheets for unexported PASS records and writes them to X:\For_Web\.
+ * Updates forweb_exported_at after successful export.
+ *
+ * Usage:
+ *   node export-datasheets.js                  Export all pending (batch mode)
+ *   node export-datasheets.js --limit 100      Export up to 100 records
+ *   node export-datasheets.js --file <paths>   Export records matching specific source files
+ *   node export-datasheets.js --serial 178439-1  Export a specific serial number
+ *   node export-datasheets.js --dry-run        Show what would be exported without writing
+ */
+
+const fs = require('fs');
+const path = require('path');
+const db = require('./db');
+
+const { loadAllSpecs, getSpecs } = require('../parsers/spec-reader');
+const { generateExactDatasheet } = require('../templates/datasheet-exact');
+
+// Configuration
+const OUTPUT_DIR = 'X:\\For_Web';
+const BATCH_SIZE = 500;
+
+async function run() {
+    const args = process.argv.slice(2);
+    const dryRun = args.includes('--dry-run');
+    const limitIdx = args.indexOf('--limit');
+    const limit = limitIdx >= 0 ? parseInt(args[limitIdx + 1]) : 0;
+    const serialIdx = args.indexOf('--serial');
+    const serial = serialIdx >= 0 ? args[serialIdx + 1] : null;
+    const fileIdx = args.indexOf('--file');
+    const files = fileIdx >= 0 ? args.slice(fileIdx + 1).filter(f => !f.startsWith('--')) : null;
+
+    console.log('========================================');
+    console.log('Datasheet Export');
+    console.log('========================================');
+    console.log(`Output: ${OUTPUT_DIR}`);
+    console.log(`Dry run: ${dryRun}`);
+    if (limit) console.log(`Limit: ${limit}`);
+    if (serial) console.log(`Serial: ${serial}`);
+    console.log(`Start: ${new Date().toISOString()}`);
+
+    if (!dryRun && !fs.existsSync(OUTPUT_DIR)) {
+        console.error(`ERROR: Output directory does not exist: ${OUTPUT_DIR}`);
+        process.exit(1);
+    }
+
+    console.log('\nLoading model specs...');
+    const specMap = loadAllSpecs();
+
+    // Build query
+    const conditions = [`overall_result = 'PASS'`, `forweb_exported_at IS NULL`];
+    const params = [];
+    let paramIdx = 0;
+
+    if (serial) {
+        paramIdx++;
+        conditions.push(`serial_number = $${paramIdx}`);
+        params.push(serial);
+    }
+
+    if (files && files.length > 0) {
+        const placeholders = files.map(() => { paramIdx++; return `$${paramIdx}`; }).join(',');
+        conditions.push(`source_file IN (${placeholders})`);
+        params.push(...files);
+    }
+
+    let sql = `SELECT * FROM test_records WHERE ${conditions.join(' AND ')} ORDER BY test_date DESC`;
+
+    if (limit) {
+        paramIdx++;
+        sql += ` LIMIT $${paramIdx}`;
+        params.push(limit);
+    }
+
+    const records = await db.query(sql, params);
+    console.log(`\nFound ${records.length} records to export`);
+
+    if (records.length === 0) {
+        console.log('Nothing to export.');
+        await db.close();
+        return { exported: 0, skipped: 0, errors: 0 };
+    }
+
+    let exported = 0;
+    let skipped = 0;
+    let errors = 0;
+    let noSpecs = 0;
+    let pendingUpdates = [];
+
+    for (const record of records) {
+        try {
+            const filename = record.serial_number + '.TXT';
+            const outputPath = path.join(OUTPUT_DIR, filename);
+
+            // VASLOG_ENG: verbatim byte-for-byte copy of the original file.
+            // Using fs.copyFileSync avoids any utf-8 round-trip that would
+            // corrupt non-ASCII bytes (CP1252 etc.) in customer datasheets.
+            // Fall back to writing raw_data if the source file is gone.
+            if (record.log_type === 'VASLOG_ENG') {
+                if (dryRun) {
+                    console.log(`  [DRY RUN] Would copy: ${record.source_file} -> ${filename}`);
+                    exported++;
+                    continue;
+                }
+                if (record.source_file && fs.existsSync(record.source_file)) {
+                    fs.copyFileSync(record.source_file, outputPath);
+                } else {
+                    console.warn(`[WARN] source file missing, writing decoded raw_data for ${record.serial_number}`);
+                    if (!record.raw_data) {
+                        skipped++;
+                        continue;
+                    }
+                    fs.writeFileSync(outputPath, record.raw_data, 'utf8');
+                }
+                pendingUpdates.push(record.id);
+                exported++;
+
+                if (pendingUpdates.length >= BATCH_SIZE) {
+                    await flushUpdates(pendingUpdates);
+                    pendingUpdates = [];
+                    process.stdout.write(`\r  Exported: ${exported} / ${records.length}`);
+                }
+                continue;
+            }
+
+            // Template-generated datasheet path.
+            const specs = getSpecs(specMap, record.model_number);
+            if (!specs) {
+                noSpecs++;
+                skipped++;
+                continue;
+            }
+            const txt = generateExactDatasheet(record, specs);
+            if (!txt) {
+                skipped++;
+                continue;
+            }
+
+            if (dryRun) {
+                console.log(`  [DRY RUN] Would write: ${filename}`);
+                exported++;
+            } else {
+                fs.writeFileSync(outputPath, txt, 'utf8');
+                pendingUpdates.push(record.id);
+                exported++;
+
+                // Batch commit
+                if (pendingUpdates.length >= BATCH_SIZE) {
+                    await flushUpdates(pendingUpdates);
+                    pendingUpdates = [];
+                    process.stdout.write(`\r  Exported: ${exported} / ${records.length}`);
+                }
+            }
+        } catch (err) {
+            console.error(`\n  ERROR exporting ${record.serial_number}: ${err.message}`);
+            errors++;
+        }
+    }
+
+    // Flush remaining updates
+    if (pendingUpdates.length > 0) {
+        await flushUpdates(pendingUpdates);
+    }
+
+    console.log(`\n\n========================================`);
+    console.log(`Export Complete`);
+    console.log(`========================================`);
+    console.log(`Exported:  ${exported}`);
+    console.log(`Skipped:   ${skipped} (${noSpecs} missing specs)`);
+    console.log(`Errors:    ${errors}`);
+    console.log(`End: ${new Date().toISOString()}`);
+
+    await db.close();
+    return { exported, skipped, errors };
+}
+
+async function flushUpdates(ids) {
+    const now = new Date().toISOString();
+    await db.transaction(async (txClient) => {
+        for (const id of ids) {
+            await txClient.execute(
+                'UPDATE test_records SET forweb_exported_at = $1 WHERE id = $2',
+                [now, id]
+            );
+        }
+    });
+}
+
+// Export function for use by import.js (no db argument -- uses shared pool)
+async function exportNewRecords(specMap, filePaths) {
+    if (!fs.existsSync(OUTPUT_DIR)) {
+        console.log(`[EXPORT] Output directory not found: ${OUTPUT_DIR}`);
+        return 0;
+    }
+
+    const conditions = [`overall_result = 'PASS'`, `forweb_exported_at IS NULL`];
+    const params = [];
+    let paramIdx = 0;
+
+    if (filePaths && filePaths.length > 0) {
+        const placeholders = filePaths.map(() => { paramIdx++; return `$${paramIdx}`; }).join(',');
+        conditions.push(`source_file IN (${placeholders})`);
+        params.push(...filePaths);
+    }
+
+    const sql = `SELECT * FROM test_records WHERE ${conditions.join(' AND ')}`;
+    const records = await db.query(sql, params);
+    if (records.length === 0) return 0;
+
+    let exported = 0;
+
+    await db.transaction(async (txClient) => {
+        for (const record of records) {
+            const filename = record.serial_number + '.TXT';
+            const outputPath = path.join(OUTPUT_DIR, filename);
+
+            try {
+                // VASLOG_ENG: verbatim copy, preserving original bytes.
+                if (record.log_type === 'VASLOG_ENG') {
+                    if (record.source_file && fs.existsSync(record.source_file)) {
+                        fs.copyFileSync(record.source_file, outputPath);
+                    } else {
+                        console.warn(`[WARN] source file missing, writing decoded raw_data for ${record.serial_number}`);
+                        if (!record.raw_data) continue;
+                        fs.writeFileSync(outputPath, record.raw_data, 'utf8');
+                    }
+                } else {
+                    const specs = getSpecs(specMap, record.model_number);
+                    if (!specs) continue;
+                    const txt = generateExactDatasheet(record, specs);
+                    if (!txt) continue;
+                    fs.writeFileSync(outputPath, txt, 'utf8');
+                }
+
+                await txClient.execute(
+                    'UPDATE test_records SET forweb_exported_at = $1 WHERE id = $2',
+                    [new Date().toISOString(), record.id]
+                );
+                exported++;
+            } catch (err) {
+                console.error(`[EXPORT] Error writing ${filename}: ${err.message}`);
+            }
+        }
+    });
+
+    console.log(`[EXPORT] Generated ${exported} datasheet(s)`);
+    return exported;
+}
+
+if (require.main === module) {
+    run().catch(console.error);
+}
+
+module.exports = { exportNewRecords };

projects/dataforth-dos/datasheet-pipeline/current-testdatadb/import.js

@@ -0,0 +1,396 @@
+/**
+ * Data Import Script
+ * Imports test data from DAT and SHT files into PostgreSQL database
+ */
+
+const fs = require('fs');
+const path = require('path');
+const db = require('./db');
+
+const { parseMultilineFile, extractTestStation } = require('../parsers/multiline');
+const { parseCsvFile } = require('../parsers/csvline');
+const { parseShtFile } = require('../parsers/shtfile');
+const { parseVaslogEngTxt } = require('../parsers/vaslog-engtxt');
+
+// Data source paths
+const TEST_PATH = 'C:/Shares/test';
+const RECOVERY_PATH = 'C:/Shares/Recovery-TEST';
+const HISTLOGS_PATH = path.join(TEST_PATH, 'Ate/HISTLOGS');
+
+// Log types and their parsers.
+// NOTE: `recursive` defaults to TRUE when absent (walk subfolders by default,
+// preserving pre-existing production behavior for DSCLOG/5BLOG/8BLOG/PWRLOG/
+// SCTLOG/7BLOG). Set it to FALSE explicitly on VASLOG so the .DAT walk does
+// NOT descend into the "VASLOG - Engineering Tested" subfolder (belt-and-
+// suspenders: the .DAT glob wouldn't match .txt, but be explicit anyway).
+// VASLOG_ENG also sets recursive:false -- the eng-tested dir is flat.
+const LOG_TYPES = {
+    'DSCLOG': { parser: 'multiline', ext: '.DAT' },
+    '5BLOG': { parser: 'multiline', ext: '.DAT' },
+    '8BLOG': { parser: 'multiline', ext: '.DAT' },
+    'PWRLOG': { parser: 'multiline', ext: '.DAT' },
+    'SCTLOG': { parser: 'multiline', ext: '.DAT' },
+    'VASLOG': { parser: 'multiline', ext: '.DAT', recursive: false },
+    '7BLOG': { parser: 'csvline', ext: '.DAT' },
+    // Engineering-tested SCMHVAS pre-rendered datasheets live under VASLOG/"VASLOG - Engineering Tested"/
+    'VASLOG_ENG': { parser: 'vaslog-engtxt', ext: '.txt', dir: 'VASLOG/VASLOG - Engineering Tested', recursive: false }
+};
+
+// Find all files of a specific type in a directory
+function findFiles(dir, pattern, recursive = true) {
+    const results = [];
+
+    try {
+        if (!fs.existsSync(dir)) return results;
+
+        const items = fs.readdirSync(dir, { withFileTypes: true });
+
+        for (const item of items) {
+            const fullPath = path.join(dir, item.name);
+
+            if (item.isDirectory() && recursive) {
+                results.push(...findFiles(fullPath, pattern, recursive));
+            } else if (item.isFile()) {
+                if (pattern.test(item.name)) {
+                    results.push(fullPath);
+                }
+            }
+        }
+    } catch (err) {
+        // Ignore permission errors
+    }
+
+    return results;
+}
+
+// Parse records from a file (sync -- file I/O only)
+function parseFile(filePath, logType, parser) {
+    const testStation = extractTestStation(filePath);
+
+    switch (parser) {
+        case 'multiline':
+            return parseMultilineFile(filePath, logType, testStation);
+        case 'csvline':
+            return parseCsvFile(filePath, testStation);
+        case 'shtfile':
+            return parseShtFile(filePath, testStation);
+        case 'vaslog-engtxt':
+            return parseVaslogEngTxt(filePath, testStation);
+        default:
+            return [];
+    }
+}
+
+// Batch insert records into PostgreSQL
+async function insertBatch(txClient, records) {
+    let imported = 0;
+    for (const record of records) {
+        try {
+            const result = await txClient.execute(
+                `INSERT INTO test_records
+                (log_type, model_number, serial_number, test_date, test_station, overall_result, raw_data, source_file)
+                VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
+                ON CONFLICT (log_type, model_number, serial_number, test_date, test_station)
+                DO UPDATE SET raw_data = EXCLUDED.raw_data, overall_result = EXCLUDED.overall_result`,
+                [
+                    record.log_type,
+                    record.model_number,
+                    record.serial_number,
+                    record.test_date,
+                    record.test_station,
+                    record.overall_result,
+                    record.raw_data,
+                    record.source_file
+                ]
+            );
+            if (result.rowCount > 0) imported++;
+        } catch (err) {
+            // Constraint error - skip
+        }
+    }
+    return imported;
+}
+
+// Import records from a file
+async function importFile(txClient, filePath, logType, parser) {
+    let records = [];
+
+    try {
+        records = parseFile(filePath, logType, parser);
+        const imported = await insertBatch(txClient, records);
+        return { total: records.length, imported };
+    } catch (err) {
+        console.error(`Error importing ${filePath}: ${err.message}`);
+        return { total: 0, imported: 0 };
+    }
+}
+
+// Import from HISTLOGS (master consolidated logs)
+async function importHistlogs(txClient) {
+    console.log('\n=== Importing from HISTLOGS ===');
+
+    let totalImported = 0;
+    let totalRecords = 0;
+
+    for (const [logType, config] of Object.entries(LOG_TYPES)) {
+        const subdir = config.dir || logType;
+        const logDir = path.join(HISTLOGS_PATH, subdir);
+
+        if (!fs.existsSync(logDir)) {
+            console.log(`  ${logType}: directory not found`);
+            continue;
+        }
+
+        const files = findFiles(logDir, new RegExp(`\\${config.ext}$`, 'i'), config.recursive !== false);
+        console.log(`  ${logType}: found ${files.length} files`);
+
+        for (const file of files) {
+            const { total, imported } = await importFile(txClient, file, logType, config.parser);
+            totalRecords += total;
+            totalImported += imported;
+        }
+    }
+
+    console.log(`  HISTLOGS total: ${totalImported} records imported (${totalRecords} parsed)`);
+    return totalImported;
+}
+
+// Import from test station logs
+async function importStationLogs(txClient, basePath, label) {
+    console.log(`\n=== Importing from ${label} ===`);
+
+    let totalImported = 0;
+    let totalRecords = 0;
+
+    const stationPattern = /^TS-\d+[LR]?$/i;
+    let stations = [];
+
+    try {
+        const items = fs.readdirSync(basePath, { withFileTypes: true });
+        stations = items
+            .filter(i => i.isDirectory() && stationPattern.test(i.name))
+            .map(i => i.name);
+    } catch (err) {
+        console.log(`  Error reading ${basePath}: ${err.message}`);
+        return 0;
+    }
+
+    console.log(`  Found stations: ${stations.join(', ')}`);
+
+    for (const station of stations) {
+        const logsDir = path.join(basePath, station, 'LOGS');
+
+        if (!fs.existsSync(logsDir)) continue;
+
+        for (const [logType, config] of Object.entries(LOG_TYPES)) {
+            const subdir = config.dir || logType;
+            const logDir = path.join(logsDir, subdir);
+
+            if (!fs.existsSync(logDir)) continue;
+
+            const files = findFiles(logDir, new RegExp(`\\${config.ext}$`, 'i'), config.recursive !== false);
+
+            for (const file of files) {
+                const { total, imported } = await importFile(txClient, file, logType, config.parser);
+                totalRecords += total;
+                totalImported += imported;
+            }
+        }
+    }
+
+    // Also import SHT files
+    const shtFiles = findFiles(basePath, /\.SHT$/i, true);
+    console.log(`  Found ${shtFiles.length} SHT files`);
+
+    for (const file of shtFiles) {
+        const { total, imported } = await importFile(txClient, file, 'SHT', 'shtfile');
+        totalRecords += total;
+        totalImported += imported;
+    }
+
+    console.log(`  ${label} total: ${totalImported} records imported (${totalRecords} parsed)`);
+    return totalImported;
+}
+
+// Import from Recovery-TEST backups (newest first)
+async function importRecoveryBackups(txClient) {
+    console.log('\n=== Importing from Recovery-TEST backups ===');
+
+    if (!fs.existsSync(RECOVERY_PATH)) {
+        console.log('  Recovery-TEST directory not found');
+        return 0;
+    }
+
+    const backups = fs.readdirSync(RECOVERY_PATH, { withFileTypes: true })
+        .filter(i => i.isDirectory() && /^\d{2}-\d{2}-\d{2}$/.test(i.name))
+        .map(i => i.name)
+        .sort()
+        .reverse();
+
+    console.log(`  Found backup dates: ${backups.join(', ')}`);
+
+    let totalImported = 0;
+
+    for (const backup of backups) {
+        const backupPath = path.join(RECOVERY_PATH, backup);
+        const imported = await importStationLogs(txClient, backupPath, `Recovery-TEST/${backup}`);
+        totalImported += imported;
+    }
+
+    return totalImported;
+}
+
+// Main import function
+async function runImport() {
+    console.log('========================================');
+    console.log('Test Data Import');
+    console.log('========================================');
+    console.log(`Start time: ${new Date().toISOString()}`);
+
+    let grandTotal = 0;
+
+    await db.transaction(async (txClient) => {
+        grandTotal += await importHistlogs(txClient);
+        grandTotal += await importRecoveryBackups(txClient);
+        grandTotal += await importStationLogs(txClient, TEST_PATH, 'test');
+    });
+
+    const stats = await db.queryOne('SELECT COUNT(*) as count FROM test_records');
+
+    console.log('\n========================================');
+    console.log('Import Complete');
+    console.log('========================================');
+    console.log(`Total records in database: ${stats.count}`);
+    console.log(`End time: ${new Date().toISOString()}`);
+
+    await db.close();
+}
+
+// Import a single file (for incremental imports from sync)
+async function importSingleFile(filePath) {
+    console.log(`Importing: ${filePath}`);
+
+    let logType = null;
+    let parser = null;
+
+    // VASLOG_ENG subpath must be checked before VASLOG (substring overlap).
+    if (filePath.includes('VASLOG - Engineering Tested')) {
+        logType = 'VASLOG_ENG';
+        parser = LOG_TYPES['VASLOG_ENG'].parser;
+    } else {
+        for (const [type, config] of Object.entries(LOG_TYPES)) {
+            if (type === 'VASLOG_ENG') continue;
+            if (filePath.includes(type)) {
+                logType = type;
+                parser = config.parser;
+                break;
+            }
+        }
+    }
+
+    if (!logType) {
+        if (/\.SHT$/i.test(filePath)) {
+            logType = 'SHT';
+            parser = 'shtfile';
+        } else {
+            console.log(`  Unknown log type for: ${filePath}`);
+            return { total: 0, imported: 0 };
+        }
+    }
+
+    let result;
+    await db.transaction(async (txClient) => {
+        result = await importFile(txClient, filePath, logType, parser);
+    });
+
+    console.log(`  Imported ${result.imported} of ${result.total} records`);
+    return result;
+}
+
+// Import multiple files (for batch incremental imports)
+async function importFiles(filePaths) {
+    console.log(`\n========================================`);
+    console.log(`Incremental Import: ${filePaths.length} files`);
+    console.log(`========================================`);
+
+    let totalImported = 0;
+    let totalRecords = 0;
+
+    await db.transaction(async (txClient) => {
+        for (const filePath of filePaths) {
+            let logType = null;
+            let parser = null;
+
+            // VASLOG_ENG subpath must be checked before the generic loop --
+            // otherwise `includes('VASLOG')` hits first and the eng .txt gets
+            // dispatched to the multiline parser. Mirror importSingleFile().
+            if (filePath.includes('VASLOG - Engineering Tested')) {
+                logType = 'VASLOG_ENG';
+                parser = LOG_TYPES['VASLOG_ENG'].parser;
+            } else {
+                for (const [type, config] of Object.entries(LOG_TYPES)) {
+                    if (type === 'VASLOG_ENG') continue;
+                    if (filePath.includes(type)) {
+                        logType = type;
+                        parser = config.parser;
+                        break;
+                    }
+                }
+            }
+
+            if (!logType) {
+                if (/\.SHT$/i.test(filePath)) {
+                    logType = 'SHT';
+                    parser = 'shtfile';
+                } else {
+                    console.log(`  Skipping unknown type: ${filePath}`);
+                    continue;
+                }
+            }
+
+            const { total, imported } = await importFile(txClient, filePath, logType, parser);
+            totalRecords += total;
+            totalImported += imported;
+            console.log(`  ${path.basename(filePath)}: ${imported}/${total} records`);
+        }
+    });
+
+    console.log(`\nTotal: ${totalImported} records imported (${totalRecords} parsed)`);
+
+    // Export datasheets for newly imported records
+    if (totalImported > 0) {
+        try {
+            const { loadAllSpecs } = require('../parsers/spec-reader');
+            const { exportNewRecords } = require('./export-datasheets');
+            const specMap = loadAllSpecs();
+            await exportNewRecords(specMap, filePaths);
+        } catch (err) {
+            console.error(`[EXPORT] Datasheet export failed: ${err.message}`);
+        }
+    }
+
+    return { total: totalRecords, imported: totalImported };
+}
+
+// Run if called directly
+if (require.main === module) {
+    const args = process.argv.slice(2);
+
+    if (args.length > 0 && args[0] === '--file') {
+        const files = args.slice(1);
+        if (files.length === 0) {
+            console.log('Usage: node import.js --file <file1> [file2] ...');
+            process.exit(1);
+        }
+        importFiles(files).then(() => db.close()).catch(console.error);
+    } else if (args.length > 0 && args[0] === '--help') {
+        console.log('Usage:');
+        console.log('  node import.js              Full import from all sources');
+        console.log('  node import.js --file <f>   Import specific file(s)');
+        process.exit(0);
+    } else {
+        runImport().catch(console.error);
+    }
+}
+
+module.exports = { runImport, importSingleFile, importFiles };

projects/dataforth-dos/datasheet-pipeline/current-testdatadb/routes/api.js

@@ -0,0 +1,487 @@
+/**
+ * API Routes for Test Data Database
+ *
+ * PostgreSQL version - uses pg.Pool via database/db.js.
+ * All route handlers are async. FTS uses tsvector/plainto_tsquery.
+ */
+
+const express = require('express');
+const path = require('path');
+const db = require('../database/db');
+const { generateDatasheet } = require('../templates/datasheet');
+
+const router = express.Router();
+
+// ---------------------------------------------------------------------------
+// Helpers
+// ---------------------------------------------------------------------------
+const MAX_LIMIT = 1000;
+
+function clampLimit(value) {
+    const parsed = parseInt(value, 10);
+    if (isNaN(parsed) || parsed < 1) return 100;
+    return Math.min(parsed, MAX_LIMIT);
+}
+
+function clampOffset(value) {
+    const parsed = parseInt(value, 10);
+    if (isNaN(parsed) || parsed < 0) return 0;
+    return parsed;
+}
+
+// ---------------------------------------------------------------------------
+// GET /api/search
+// Search test records
+// Query params: serial, model, from, to, result, q, station, logtype, limit, offset
+// ---------------------------------------------------------------------------
+router.get('/search', async (req, res) => {
+    try {
+        const { serial, model, from, to, result, q, station, logtype, workorder } = req.query;
+        const limit = clampLimit(req.query.limit || 100);
+        const offset = clampOffset(req.query.offset || 0);
+
+        const conditions = [];
+        const params = [];
+        let paramIdx = 0;
+
+        const addParam = (val) => {
+            paramIdx++;
+            params.push(val);
+            return '$' + paramIdx;
+        };
+
+        if (q) {
+            // Full-text search using tsvector
+            conditions.push(`search_vector @@ plainto_tsquery('english', ${addParam(q)})`);
+        }
+
+        if (serial) {
+            const val = serial.includes('%') ? serial : `%${serial}%`;
+            conditions.push(`serial_number LIKE ${addParam(val)}`);
+        }
+
+        if (workorder) {
+            conditions.push(`work_order = ${addParam(workorder)}`);
+        }
+
+        if (model) {
+            const val = model.includes('%') ? model : `%${model}%`;
+            conditions.push(`model_number LIKE ${addParam(val)}`);
+        }
+
+        if (from) {
+            conditions.push(`test_date >= ${addParam(from)}`);
+        }
+
+        if (to) {
+            conditions.push(`test_date <= ${addParam(to)}`);
+        }
+
+        if (result) {
+            conditions.push(`overall_result = ${addParam(result.toUpperCase())}`);
+        }
+
+        if (station) {
+            conditions.push(`test_station = ${addParam(station)}`);
+        }
+
+        if (logtype) {
+            conditions.push(`log_type = ${addParam(logtype)}`);
+        }
+
+        const where = conditions.length > 0 ? 'WHERE ' + conditions.join(' AND ') : '';
+
+        const dataSql = `SELECT * FROM test_records ${where} ORDER BY test_date DESC, serial_number LIMIT ${addParam(limit)} OFFSET ${addParam(offset)}`;
+        const countSql = `SELECT COUNT(*) as count FROM test_records ${where}`;
+        const countParams = params.slice(0, paramIdx - 2); // exclude limit/offset
+
+        const [records, countRow] = await Promise.all([
+            db.query(dataSql, params),
+            db.queryOne(countSql, countParams),
+        ]);
+
+        res.json({
+            records,
+            total: countRow?.count ? parseInt(countRow.count, 10) : records.length,
+            limit,
+            offset
+        });
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [SEARCH ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/record/:id
+// Get single record by ID
+// ---------------------------------------------------------------------------
+router.get('/record/:id', async (req, res) => {
+    try {
+        const record = await db.queryOne('SELECT * FROM test_records WHERE id = $1', [req.params.id]);
+
+        if (!record) {
+            return res.status(404).json({ error: 'Record not found' });
+        }
+
+        res.json(record);
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [RECORD ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/datasheet/:id
+// Generate datasheet for a record
+// Query params: format (html, txt)
+// ---------------------------------------------------------------------------
+router.get('/datasheet/:id', async (req, res) => {
+    try {
+        const record = await db.queryOne('SELECT * FROM test_records WHERE id = $1', [req.params.id]);
+
+        if (!record) {
+            return res.status(404).json({ error: 'Record not found' });
+        }
+
+        const format = req.query.format || 'html';
+
+        // Try exact-match formatter first
+        const { loadAllSpecs, getSpecs } = require('../parsers/spec-reader');
+        const { generateExactDatasheet } = require('../templates/datasheet-exact');
+        const specMap = loadAllSpecs();
+        const specs = getSpecs(specMap, record.model_number);
+        const exactTxt = generateExactDatasheet(record, specs);
+
+        if (exactTxt && format === 'html') {
+            // Render exact-match TXT as styled HTML page
+            const escaped = exactTxt
+                .replace(/&/g, '&amp;')
+                .replace(/</g, '&lt;')
+                .replace(/>/g, '&gt;');
+            const html = `<!DOCTYPE html>
+<html>
+<head>
+    <title>Test Data Sheet - ${record.serial_number}</title>
+    <style>
+        body {
+            margin: 0;
+            padding: 20px;
+            background: #f0f0f0;
+            display: flex;
+            justify-content: center;
+        }
+        .page {
+            background: white;
+            padding: 40px 30px;
+            max-width: 720px;
+            width: 100%;
+            box-shadow: 0 2px 8px rgba(0,0,0,0.15);
+            border: 1px solid #ccc;
+        }
+        pre {
+            font-family: 'Courier New', Courier, monospace;
+            font-size: 11px;
+            line-height: 1.4;
+            margin: 0;
+            white-space: pre;
+            overflow-x: auto;
+        }
+        .toolbar {
+            position: fixed;
+            top: 10px;
+            right: 10px;
+            display: flex;
+            gap: 8px;
+        }
+        .toolbar button {
+            padding: 8px 16px;
+            border: 1px solid #999;
+            background: white;
+            cursor: pointer;
+            font-size: 13px;
+            border-radius: 4px;
+        }
+        .toolbar button:hover { background: #e0e0e0; }
+        @media print {
+            body { background: white; padding: 0; }
+            .page { box-shadow: none; border: none; padding: 0; }
+            .toolbar { display: none; }
+        }
+    </style>
+</head>
+<body>
+    <div class="toolbar">
+        <button onclick="window.print()">Print</button>
+        <button onclick="window.open('/api/datasheet/${record.id}/pdf')">Download PDF</button>
+        <button onclick="window.close()">Close</button>
+    </div>
+    <div class="page">
+        <pre>${escaped}</pre>
+    </div>
+</body>
+</html>`;
+            res.type('html').send(html);
+        } else if (exactTxt && format === 'txt') {
+            res.type('text/plain').send(exactTxt);
+        } else {
+            // Fall back to generic template
+            const datasheet = generateDatasheet(record, format);
+            if (format === 'html') {
+                res.type('html').send(datasheet);
+            } else {
+                res.type('text/plain').send(datasheet);
+            }
+        }
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [DATASHEET ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/datasheet/:id/pdf
+// Generate PDF datasheet for a record (on-demand download)
+// ---------------------------------------------------------------------------
+router.get('/datasheet/:id/pdf', async (req, res) => {
+    try {
+        const record = await db.queryOne('SELECT * FROM test_records WHERE id = $1', [req.params.id]);
+
+        if (!record) {
+            return res.status(404).json({ error: 'Record not found' });
+        }
+
+        const { loadAllSpecs, getSpecs } = require('../parsers/spec-reader');
+        const { generateExactDatasheet } = require('../templates/datasheet-exact');
+        const PDFDocument = require('pdfkit');
+
+        const specMap = loadAllSpecs();
+        const specs = getSpecs(specMap, record.model_number);
+        let txt = generateExactDatasheet(record, specs);
+
+        // Fall back to generic datasheet if exact-match formatter doesn't support this family
+        if (!txt) {
+            txt = generateDatasheet(record, 'txt');
+        }
+
+        if (!txt) {
+            return res.status(422).json({ error: 'Could not generate datasheet (missing specs or data)' });
+        }
+
+        const doc = new PDFDocument({
+            size: 'LETTER',
+            margins: { top: 36, bottom: 36, left: 36, right: 36 }
+        });
+
+        res.setHeader('Content-Type', 'application/pdf');
+        res.setHeader('Content-Disposition', `attachment; filename="${record.serial_number}.pdf"`);
+        doc.pipe(res);
+
+        doc.font('Courier').fontSize(9.5);
+        const lines = txt.split(/\r?\n/);
+        for (const line of lines) {
+            doc.text(line, { lineGap: 1 });
+        }
+
+        doc.end();
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [PDF ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/stats
+// Get database statistics
+// ---------------------------------------------------------------------------
+router.get('/stats', async (req, res) => {
+    try {
+        const [totalRow, byLogType, byResult, byStation, dateRange, recentSerials] = await Promise.all([
+            db.queryOne('SELECT COUNT(*) as count FROM test_records'),
+            db.query('SELECT log_type, COUNT(*) as count FROM test_records GROUP BY log_type ORDER BY count DESC'),
+            db.query('SELECT overall_result, COUNT(*) as count FROM test_records GROUP BY overall_result'),
+            db.query(`SELECT test_station, COUNT(*) as count FROM test_records
+                      WHERE test_station IS NOT NULL AND test_station != ''
+                      GROUP BY test_station ORDER BY test_station`),
+            db.queryOne('SELECT MIN(test_date) as oldest, MAX(test_date) as newest FROM test_records'),
+            db.query(`SELECT DISTINCT serial_number, model_number, test_date
+                      FROM test_records ORDER BY test_date DESC LIMIT 10`),
+        ]);
+
+        res.json({
+            total_records: parseInt(totalRow.count, 10),
+            by_log_type: byLogType.map(r => ({ ...r, count: parseInt(r.count, 10) })),
+            by_result: byResult.map(r => ({ ...r, count: parseInt(r.count, 10) })),
+            by_station: byStation.map(r => ({ ...r, count: parseInt(r.count, 10) })),
+            date_range: dateRange,
+            recent_serials: recentSerials,
+        });
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [STATS ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/filters
+// Get available filter options (test stations, log types, models)
+// ---------------------------------------------------------------------------
+router.get('/filters', async (req, res) => {
+    try {
+        const [stations, logTypes, models] = await Promise.all([
+            db.query(`SELECT DISTINCT test_station FROM test_records
+                      WHERE test_station IS NOT NULL AND test_station != ''
+                      ORDER BY test_station`),
+            db.query('SELECT DISTINCT log_type FROM test_records ORDER BY log_type'),
+            db.query(`SELECT DISTINCT model_number, COUNT(*) as count FROM test_records
+                      GROUP BY model_number ORDER BY count DESC LIMIT 500`),
+        ]);
+
+        res.json({
+            stations: stations.map(r => r.test_station),
+            log_types: logTypes.map(r => r.log_type),
+            models: models.map(r => ({ ...r, count: parseInt(r.count, 10) })),
+        });
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [FILTERS ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/export
+// Export search results as CSV
+// ---------------------------------------------------------------------------
+router.get('/export', async (req, res) => {
+    try {
+        const { serial, model, from, to, result, station, logtype } = req.query;
+
+        const conditions = [];
+        const params = [];
+        let paramIdx = 0;
+
+        const addParam = (val) => {
+            paramIdx++;
+            params.push(val);
+            return '$' + paramIdx;
+        };
+
+        if (serial) {
+            const val = serial.includes('%') ? serial : `%${serial}%`;
+            conditions.push(`serial_number LIKE ${addParam(val)}`);
+        }
+
+        if (model) {
+            const val = model.includes('%') ? model : `%${model}%`;
+            conditions.push(`model_number LIKE ${addParam(val)}`);
+        }
+
+        if (from) {
+            conditions.push(`test_date >= ${addParam(from)}`);
+        }
+
+        if (to) {
+            conditions.push(`test_date <= ${addParam(to)}`);
+        }
+
+        if (result) {
+            conditions.push(`overall_result = ${addParam(result.toUpperCase())}`);
+        }
+
+        if (station) {
+            conditions.push(`test_station = ${addParam(station)}`);
+        }
+
+        if (logtype) {
+            conditions.push(`log_type = ${addParam(logtype)}`);
+        }
+
+        const where = conditions.length > 0 ? 'WHERE ' + conditions.join(' AND ') : '';
+        const sql = `SELECT * FROM test_records ${where} ORDER BY test_date DESC, serial_number LIMIT 10000`;
+
+        const records = await db.query(sql, params);
+
+        // Generate CSV
+        const headers = ['id', 'log_type', 'model_number', 'serial_number', 'test_date', 'test_station', 'overall_result', 'source_file'];
+        let csv = headers.join(',') + '\n';
+
+        for (const record of records) {
+            const row = headers.map(h => {
+                const val = record[h] || '';
+                return `"${String(val).replace(/"/g, '""')}"`;
+            });
+            csv += row.join(',') + '\n';
+        }
+
+        res.setHeader('Content-Type', 'text/csv');
+        res.setHeader('Content-Disposition', 'attachment; filename=test_records.csv');
+        res.send(csv);
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [EXPORT ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/workorder/:wo
+// Get work order details and all associated test lines
+// ---------------------------------------------------------------------------
+router.get('/workorder/:wo', async (req, res) => {
+    try {
+        const wo = req.params.wo;
+
+        const [header, lines, testRecords] = await Promise.all([
+            db.queryOne('SELECT * FROM work_orders WHERE wo_number = $1', [wo]),
+            db.query('SELECT * FROM work_order_lines WHERE wo_number = $1 ORDER BY test_date, test_time', [wo]),
+            db.query(
+                'SELECT id, log_type, model_number, serial_number, test_date, test_station, overall_result, work_order FROM test_records WHERE work_order = $1 ORDER BY serial_number',
+                [wo]
+            ),
+        ]);
+
+        res.json({
+            work_order: header || { wo_number: wo },
+            lines,
+            test_records: testRecords,
+        });
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [WO ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/workorder-search?q=<query>
+// Search work orders by number (prefix match)
+// ---------------------------------------------------------------------------
+router.get('/workorder-search', async (req, res) => {
+    try {
+        const q = req.query.q || '';
+        if (q.length < 2) {
+            return res.json({ results: [] });
+        }
+
+        const results = await db.query(
+            'SELECT wo_number, wo_date, program, test_station FROM work_orders WHERE wo_number LIKE $1 ORDER BY wo_date DESC LIMIT 50',
+            [q + '%']
+        );
+
+        res.json({ results });
+    } catch (err) {
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// Cleanup function for graceful shutdown
+// ---------------------------------------------------------------------------
+async function cleanup() {
+    try {
+        await db.close();
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [CLEANUP ERROR] ${err.message}`);
+    }
+}
+
+module.exports = router;
+module.exports.cleanup = cleanup;

projects/dataforth-dos/datasheet-pipeline/deploy_ui_feature.py

@@ -0,0 +1,115 @@
+"""Deploy the api_uploaded_at + UI push feature to AD2 in the correct order:
+
+  1. SFTP server_inventory.txt to AD2 (one-time, for back-population)
+  2. SFTP migration SQL + back-populate script
+  3. Run migration via psql
+  4. Run back-populate
+  5. Backup current production files (import.js already backed up earlier this
+     session; backup routes/api.js + public/index.html + database/upload-to-api.js)
+  6. SFTP updated upload-to-api.js, routes/api.js, public/index.html
+  7. node --check on AD2
+  8. Restart testdatadb service
+  9. Verify
+"""
+import base64, paramiko, subprocess, time, yaml, os
+
+pwd_raw = yaml.safe_load(subprocess.run(['sops','-d','D:/vault/clients/dataforth/ad2.sops.yaml'],
+    capture_output=True, text=True, timeout=30, check=True).stdout)['credentials']['password']
+PWD = pwd_raw  # vault now has clean password
+
+LOCAL_IMPL = r'D:\claudetools\projects\dataforth-dos\datasheet-pipeline\implementation-upload'
+REMOTE_DB  = 'C:/Shares/testdatadb/database'
+REMOTE_API = 'C:/Shares/testdatadb/routes'
+REMOTE_WEB = 'C:/Shares/testdatadb/public'
+PROD_DIR   = 'C:/ProgramData/dataforth-uploader'
+SERVER_INV_LOCAL = r'C:\Users\guru\AppData\Local\Temp\server_inventory.txt'
+
+c = paramiko.SSHClient(); c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+c.connect('192.168.0.6', username='sysadmin', password=PWD,
+          timeout=30, banner_timeout=45, look_for_keys=False, allow_agent=False)
+
+def ps(cmd, to=120):
+    enc = base64.b64encode(cmd.encode('utf-16-le')).decode()
+    _, o, e = c.exec_command(f'powershell -NoProfile -EncodedCommand {enc}', timeout=to)
+    return o.read().decode('utf-8','replace'), e.read().decode('utf-8','replace')
+
+print('[1] SFTP server_inventory.txt to AD2 (for back-population)')
+sftp = c.open_sftp()
+sftp.put(SERVER_INV_LOCAL, f'{PROD_DIR}/server_inventory.txt')
+sftp.close()
+out, _ = ps(f'$f = "{PROD_DIR.replace(chr(47),chr(92))}\\server_inventory.txt"; "bytes: $((Get-Item $f).Length)"; "lines: $((Get-Content $f).Count)"')
+print(out.rstrip())
+
+print('\n[2] SFTP migration SQL + back-populate script + new files')
+sftp = c.open_sftp()
+uploads = [
+    (f'{LOCAL_IMPL}\\database\\migrate-add-api-uploaded.sql', f'{REMOTE_DB}/migrate-add-api-uploaded.sql'),
+    (f'{LOCAL_IMPL}\\database\\back-populate-api-uploaded.js', f'{REMOTE_DB}/back-populate-api-uploaded.js'),
+    (f'{LOCAL_IMPL}\\database\\upload-to-api.js',  f'{REMOTE_DB}/upload-to-api.js'),
+    (f'{LOCAL_IMPL}\\routes\\api.js',              f'{REMOTE_API}/api.js'),
+    (f'{LOCAL_IMPL}\\public\\index.html',          f'{REMOTE_WEB}/index.html'),
+]
+# Backups first
+for _, remote_dst in uploads:
+    if remote_dst.endswith('.sql') or remote_dst.endswith('back-populate-api-uploaded.js'):
+        continue  # new file, no backup needed
+    bak = remote_dst + f'.bak-{time.strftime("%Y%m%d-%H%M%S")}'
+    try:
+        with sftp.open(remote_dst, 'rb') as src, sftp.open(bak, 'wb') as dst:
+            dst.write(src.read())
+        print(f'  backed up {remote_dst} -> {bak}')
+    except Exception as e:
+        print(f'  backup skip {remote_dst}: {e}')
+# Uploads
+for local_src, remote_dst in uploads:
+    sftp.put(local_src, remote_dst)
+    print(f'  uploaded {local_src} -> {remote_dst}')
+sftp.close()
+
+print('\n[3] run migration via psql (env DATABASE_URL expected in service context; use psql -U testdatadb if set up)')
+# check db.js to understand connection info
+out, _ = ps(r'Get-Content "C:\Shares\testdatadb\database\db.js" | Select-String "host|user|database|port|connectionString" | Select -First 10 | Out-String')
+print(out.rstrip())
+
+print('\n[3b] run migration via psql using .env creds')
+out, _ = ps(r'$env_file = "C:\Shares\testdatadb\.env"; if (Test-Path $env_file) { Get-Content $env_file } else { "no .env" }')
+print(out.rstrip())
+
+# Try discovering via the db.js defaults + running migration with Node (safer than psql here)
+out, _ = ps(
+    f'cd "{REMOTE_DB.replace("/","\\")}"; '
+    r'& node -e "const db = require(''./db''); (async () => { '
+    r'const sql = require(''fs'').readFileSync(''./migrate-add-api-uploaded.sql'', ''utf8''); '
+    r'await db.execute(sql); console.log(''[MIG OK]''); '
+    r'const c = await db.queryOne(\"SELECT COUNT(*) as c FROM information_schema.columns WHERE table_name=''test_records'' AND column_name=''api_uploaded_at''\"); '
+    r'console.log(''column exists:'' + c.c); await db.close(); })().catch(e => { console.error(''[MIG FAIL]'', e.message); process.exit(1); });" 2>&1'
+, to=60)
+print(out.rstrip())
+
+print('\n[4] run back-populate (batch 1000)')
+out, _ = ps(
+    f'cd "{REMOTE_DB.replace("/","\\")}"; '
+    f'& node back-populate-api-uploaded.js --inventory "{PROD_DIR.replace(chr(47),chr(92))}\\server_inventory.txt" --batch 1000 2>&1'
+, to=1200)
+print(out.rstrip())
+
+print('\n[5] node --check updated files')
+out, _ = ps(
+    f'cd "{REMOTE_DB.replace("/","\\")}"; & node --check upload-to-api.js; '
+    f'cd "{REMOTE_API.replace("/","\\")}"; & node --check api.js; echo "[OK]"'
+, to=60)
+print(out.rstrip())
+
+print('\n[6] restart testdatadb')
+out, _ = ps('Restart-Service testdatadb; Start-Sleep 3; Get-Service testdatadb | Select Name,Status | Format-Table -AutoSize | Out-String', to=60)
+print(out.rstrip())
+
+print('\n[7] verify API')
+out, _ = ps(
+    r'try { $r = Invoke-WebRequest "http://localhost:3000/api/stats" -UseBasicParsing -TimeoutSec 15; "GET /api/stats HTTP $($r.StatusCode)" } catch { "GET /api/stats FAIL: $_" }; '
+    r'try { $r = Invoke-WebRequest "http://localhost:3000/api/search?limit=1" -UseBasicParsing -TimeoutSec 15; "GET /api/search HTTP $($r.StatusCode)"; $j = $r.Content | ConvertFrom-Json; "first record keys: $($j.records[0].PSObject.Properties.Name -join '', '')" } catch { "GET /api/search FAIL: $_" }'
+, to=30)
+print(out.rstrip())
+
+c.close()
+print('\n[OK] deploy complete')

projects/dataforth-dos/datasheet-pipeline/deploy_upload_integration.py

@@ -0,0 +1,82 @@
+"""Deploy the testdatadb upload integration to AD2 + convert scheduled task from hourly -> daily.
+
+Steps:
+ 1. Backup current C:\\Shares\\testdatadb\\database\\import.js on AD2
+ 2. SFTP new upload-to-api.js + updated import.js
+ 3. node --check both on AD2 to be safe
+ 4. Restart testdatadb service to reload
+ 5. Re-register DataforthTestDatasheetUploader task as DAILY (was hourly)
+ 6. Verify task definition + show next run
+"""
+import base64, paramiko, subprocess, time, yaml
+
+pwd_raw = yaml.safe_load(subprocess.run(['sops','-d','D:/vault/clients/dataforth/ad2.sops.yaml'],
+    capture_output=True, text=True, timeout=30, check=True).stdout)['credentials']['password']
+PWD = pwd_raw  # Vault has been fixed — no more `.replace('\\','')` needed
+
+LOCAL = r'D:\claudetools\projects\dataforth-dos\datasheet-pipeline\implementation-upload\database'
+REMOTE = 'C:/Shares/testdatadb/database'
+
+c = paramiko.SSHClient(); c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+c.connect('192.168.0.6', username='sysadmin', password=PWD,
+          timeout=30, banner_timeout=45, look_for_keys=False, allow_agent=False)
+
+def ps(cmd, to=120):
+    enc = base64.b64encode(cmd.encode('utf-16-le')).decode()
+    _, o, e = c.exec_command(f'powershell -NoProfile -EncodedCommand {enc}', timeout=to)
+    return o.read().decode('utf-8','replace'), e.read().decode('utf-8','replace')
+
+print('[1] backup import.js on AD2')
+out, _ = ps(f'Copy-Item -LiteralPath "{REMOTE.replace("/","\\")}\\import.js" -Destination "{REMOTE.replace("/","\\")}\\import.js.bak-$(Get-Date -Format yyyyMMdd-HHmmss)" -Force; Get-ChildItem "{REMOTE.replace("/","\\")}" -Filter "import.js*" | Select Name,Length | Format-Table -AutoSize | Out-String')
+print(out.rstrip())
+
+print('\n[2] SFTP updated files')
+sftp = c.open_sftp()
+sftp.put(f'{LOCAL}/upload-to-api.js', f'{REMOTE}/upload-to-api.js')
+sftp.put(f'{LOCAL}/import.js', f'{REMOTE}/import.js')
+sftp.close()
+out, _ = ps(f'Get-ChildItem "{REMOTE.replace("/","\\")}" -Filter "upload-to-api.js","import.js" | Select Name,Length | Format-Table -AutoSize | Out-String')
+print(out.rstrip())
+
+print('\n[3] node --check on both')
+out, err = ps(f'cd "{REMOTE.replace("/","\\")}"; & node --check upload-to-api.js 2>&1; echo "---"; & node --check import.js 2>&1; echo "---end"')
+print(out.rstrip())
+if err.strip() and 'CLIXML' not in err: print('[stderr]', err[:300])
+
+print('\n[4] restart testdatadb service')
+out, err = ps('Restart-Service testdatadb; Start-Sleep 3; Get-Service testdatadb | Select Name,Status | Format-Table -AutoSize | Out-String', to=60)
+print(out.rstrip())
+if err.strip() and 'CLIXML' not in err: print('[stderr]', err[:300])
+
+print('\n[5] re-register scheduled task as DAILY (was hourly)')
+REG = r'''
+$taskName = 'DataforthTestDatasheetUploader'
+$scriptPath = 'C:\ProgramData\dataforth-uploader\run-pipeline.ps1'
+Unregister-ScheduledTask -TaskName $taskName -Confirm:$false -ErrorAction SilentlyContinue | Out-Null
+
+$argStr = '-NoProfile -ExecutionPolicy Bypass -File ' + '"' + $scriptPath + '"'
+$action = New-ScheduledTaskAction -Execute 'powershell.exe' -Argument $argStr -WorkingDirectory 'C:\ProgramData\dataforth-uploader'
+# Daily at 02:30 server time (quiet hours)
+$trigger = New-ScheduledTaskTrigger -Daily -At (Get-Date -Hour 2 -Minute 30 -Second 0)
+$principal = New-ScheduledTaskPrincipal -UserId 'SYSTEM' -LogonType ServiceAccount -RunLevel Highest
+$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -StartWhenAvailable -ExecutionTimeLimit (New-TimeSpan -Minutes 30)
+Register-ScheduledTask -TaskName $taskName -Action $action -Trigger $trigger -Principal $principal -Settings $settings -Description 'Dataforth Test Datasheet Uploader daily fallback (primary path is import.js post-export hook)' | Out-Null
+
+Write-Host '=== registered task ==='
+(Get-ScheduledTask -TaskName $taskName).Triggers | Format-List
+(Get-ScheduledTask -TaskName $taskName).Actions | Format-List
+Write-Host '=== next run ==='
+Get-ScheduledTaskInfo -TaskName $taskName | Select LastRunTime,LastTaskResult,NextRunTime | Format-List
+'''
+# Write to file on AD2, run it
+sftp = c.open_sftp()
+with sftp.open('C:/ProgramData/dataforth-uploader/register-daily.ps1', 'w') as fh:
+    fh.write(REG)
+sftp.close()
+_, o, e = c.exec_command(r'powershell -NoProfile -ExecutionPolicy Bypass -File "C:\ProgramData\dataforth-uploader\register-daily.ps1"', timeout=60)
+print(o.read().decode('utf-8','replace'))
+err = e.read().decode('utf-8','replace')
+if err.strip() and 'CLIXML' not in err: print('[stderr]', err[:300])
+
+c.close()
+print('\n[OK] deploy complete')

projects/dataforth-dos/datasheet-pipeline/implementation-upload/database/_sanity_check.js

@@ -0,0 +1,79 @@
+const fs = require('fs');
+const https = require('https');
+const { URL } = require('url');
+const db = require('./db');
+const CREDS = JSON.parse(fs.readFileSync('C:/ProgramData/dataforth-uploader/credentials.json', 'utf8'));
+
+function req(method, uri, headers) {
+    return new Promise((res, rej) => {
+        const u = new URL(uri);
+        const r = https.request({
+            hostname: u.hostname, port: u.port || 443, path: u.pathname + u.search,
+            method, headers, timeout: 20000,
+        }, rs => {
+            let d = '';
+            rs.on('data', c => d += c);
+            rs.on('end', () => res({ status: rs.statusCode, body: d }));
+        });
+        const t = setTimeout(() => { r.destroy(); rej(new Error('timeout')); }, 20000);
+        r.on('error', rej);
+        r.on('close', () => clearTimeout(t));
+        r.end();
+    });
+}
+
+(async () => {
+    const form = 'grant_type=client_credentials&client_id=' + encodeURIComponent(CREDS.CF_CLIENT_ID) +
+        '&client_secret=' + encodeURIComponent(CREDS.CF_CLIENT_SECRET) + '&scope=' + encodeURIComponent(CREDS.CF_SCOPE);
+    const tokR = await new Promise((r, j) => {
+        const u = new URL(CREDS.CF_TOKEN_URL);
+        const rq = https.request({
+            hostname: u.hostname, port: 443, path: u.pathname, method: 'POST',
+            headers: { 'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': Buffer.byteLength(form) },
+        }, rs => {
+            let d = '';
+            rs.on('data', c => d += c);
+            rs.on('end', () => r({ status: rs.statusCode, body: d }));
+        });
+        rq.on('error', j);
+        rq.write(form);
+        rq.end();
+    });
+    const token = JSON.parse(tokR.body).access_token;
+
+    async function sample(label, sql, expect) {
+        console.log('=== ' + label + ' ===');
+        const rows = await db.query(sql);
+        let hit = 0, miss = 0, err = 0;
+        for (const r of rows) {
+            try {
+                const rr = await req('GET',
+                    CREDS.CF_API_BASE + '/api/v1/TestReportDataFiles/' + encodeURIComponent(r.serial_number),
+                    { 'Authorization': 'Bearer ' + token });
+                if (rr.status === 200) hit++;
+                else if (rr.status === 404) miss++;
+                else { err++; console.log('  HTTP ' + rr.status + ' ' + r.serial_number); }
+            } catch (e) { err++; console.log('  ERR ' + r.serial_number + ' ' + e.message); }
+        }
+        console.log('  hit=' + hit + ' miss=' + miss + ' err=' + err + '  (' + expect + ')');
+        return { hit, miss, err };
+    }
+
+    await sample(
+        'Sample 1: 100 random stamped api_uploaded_at IS NOT NULL',
+        "SELECT serial_number FROM test_records WHERE api_uploaded_at IS NOT NULL ORDER BY random() LIMIT 100",
+        'expect hit=100',
+    );
+    await sample(
+        'Sample 2: 100 random unpushable PASS (NULL api_uploaded_at, PASS)',
+        "SELECT serial_number FROM test_records WHERE api_uploaded_at IS NULL AND overall_result='PASS' ORDER BY random() LIMIT 100",
+        'expect mostly miss (these are the 10K unpushables)',
+    );
+    await sample(
+        'Sample 3: 50 random FAIL',
+        "SELECT serial_number FROM test_records WHERE overall_result='FAIL' ORDER BY random() LIMIT 50",
+        'expect miss=50 (FAILs never reach Hoffman)',
+    );
+
+    await db.close();
+})().catch(e => { console.error('FATAL', e.message); process.exit(1); });

projects/dataforth-dos/datasheet-pipeline/implementation-upload/database/back-populate-api-uploaded.js

@@ -0,0 +1,74 @@
+/**
+ * One-time back-population of api_uploaded_at from server_inventory.txt.
+ *
+ * Reads SN list, UPDATEs test_records.api_uploaded_at = NOW() in batches
+ * for records whose serial_number appears in the inventory.
+ *
+ * Usage: node back-populate-api-uploaded.js [--inventory path] [--batch 1000] [--dry-run]
+ */
+const fs = require('fs');
+const db = require('./db');
+
+const args = process.argv.slice(2);
+const arg = (n, d) => { const i = args.indexOf(n); return i >= 0 ? args[i+1] : d; };
+const flag = n => args.includes(n);
+
+const INVENTORY = arg('--inventory', 'C:\\ProgramData\\dataforth-uploader\\server_inventory.txt');
+const BATCH = parseInt(arg('--batch', '1000'), 10);
+const DRY = flag('--dry-run');
+
+async function main() {
+    if (!fs.existsSync(INVENTORY)) {
+        console.error(`[FAIL] inventory not found: ${INVENTORY}`);
+        process.exit(1);
+    }
+
+    const data = fs.readFileSync(INVENTORY, 'utf8');
+    const sns = data.split(/\r?\n/).map(s => s.trim()).filter(Boolean);
+    console.log(`[INFO] inventory: ${sns.length} serial numbers`);
+    console.log(`[INFO] batch size: ${BATCH}  dry-run: ${DRY}`);
+
+    const t0 = Date.now();
+    let totalMatched = 0;
+
+    for (let i = 0; i < sns.length; i += BATCH) {
+        const chunk = sns.slice(i, i + BATCH);
+        const placeholders = chunk.map((_, j) => `$${j + 1}`).join(',');
+        if (DRY) {
+            const row = await db.queryOne(
+                `SELECT COUNT(*) as c FROM test_records WHERE serial_number IN (${placeholders}) AND api_uploaded_at IS NULL`,
+                chunk,
+            );
+            totalMatched += parseInt(row.c, 10) || 0;
+        } else {
+            const result = await db.execute(
+                `UPDATE test_records SET api_uploaded_at = NOW() WHERE serial_number IN (${placeholders}) AND api_uploaded_at IS NULL`,
+                chunk,
+            );
+            totalMatched += result.rowCount || 0;
+        }
+        if ((i / BATCH) % 20 === 0) {
+            const rate = (i + chunk.length) / Math.max(1, (Date.now() - t0) / 1000);
+            const eta = Math.round((sns.length - i - chunk.length) / Math.max(1, rate));
+            console.log(`  progress ${i + chunk.length}/${sns.length}  matched-so-far=${totalMatched}  rate=${rate.toFixed(0)}/s  eta=${eta}s`);
+        }
+    }
+
+    const elapsed = ((Date.now() - t0) / 1000).toFixed(1);
+    console.log(`\n[DONE] ${elapsed}s`);
+    console.log(`  inventory size: ${sns.length}`);
+    console.log(`  ${DRY ? 'would update' : 'updated'}: ${totalMatched}`);
+
+    // Sanity: how many records have api_uploaded_at set vs null?
+    const tot = await db.queryOne(`SELECT COUNT(*) as c FROM test_records`);
+    const set = await db.queryOne(`SELECT COUNT(*) as c FROM test_records WHERE api_uploaded_at IS NOT NULL`);
+    const nul = await db.queryOne(`SELECT COUNT(*) as c FROM test_records WHERE api_uploaded_at IS NULL`);
+    console.log(`\n[DB STATE]`);
+    console.log(`  total records:         ${tot.c}`);
+    console.log(`  api_uploaded_at SET:   ${set.c}`);
+    console.log(`  api_uploaded_at NULL:  ${nul.c}`);
+
+    await db.close();
+}
+
+main().catch(e => { console.error('[FATAL]', e); process.exit(1); });

projects/dataforth-dos/datasheet-pipeline/implementation-upload/database/export-datasheets.js

@@ -0,0 +1,257 @@
+/**
+ * Export Datasheets
+ *
+ * Generates TXT datasheets for unexported PASS records and writes them to X:\For_Web\.
+ * Updates forweb_exported_at after successful export.
+ *
+ * Usage:
+ *   node export-datasheets.js                  Export all pending (batch mode)
+ *   node export-datasheets.js --limit 100      Export up to 100 records
+ *   node export-datasheets.js --file <paths>   Export records matching specific source files
+ *   node export-datasheets.js --serial 178439-1  Export a specific serial number
+ *   node export-datasheets.js --dry-run        Show what would be exported without writing
+ */
+
+const fs = require('fs');
+const path = require('path');
+const db = require('./db');
+
+const { loadAllSpecs, getSpecs } = require('../parsers/spec-reader');
+const { generateExactDatasheet } = require('../templates/datasheet-exact');
+
+// Configuration
+const OUTPUT_DIR = 'X:\\For_Web';
+const BATCH_SIZE = 500;
+
+async function run() {
+    const args = process.argv.slice(2);
+    const dryRun = args.includes('--dry-run');
+    const limitIdx = args.indexOf('--limit');
+    const limit = limitIdx >= 0 ? parseInt(args[limitIdx + 1]) : 0;
+    const serialIdx = args.indexOf('--serial');
+    const serial = serialIdx >= 0 ? args[serialIdx + 1] : null;
+    const fileIdx = args.indexOf('--file');
+    const files = fileIdx >= 0 ? args.slice(fileIdx + 1).filter(f => !f.startsWith('--')) : null;
+
+    console.log('========================================');
+    console.log('Datasheet Export');
+    console.log('========================================');
+    console.log(`Output: ${OUTPUT_DIR}`);
+    console.log(`Dry run: ${dryRun}`);
+    if (limit) console.log(`Limit: ${limit}`);
+    if (serial) console.log(`Serial: ${serial}`);
+    console.log(`Start: ${new Date().toISOString()}`);
+
+    if (!dryRun && !fs.existsSync(OUTPUT_DIR)) {
+        console.error(`ERROR: Output directory does not exist: ${OUTPUT_DIR}`);
+        process.exit(1);
+    }
+
+    console.log('\nLoading model specs...');
+    const specMap = loadAllSpecs();
+
+    // Build query
+    const conditions = [`overall_result = 'PASS'`, `forweb_exported_at IS NULL`];
+    const params = [];
+    let paramIdx = 0;
+
+    if (serial) {
+        paramIdx++;
+        conditions.push(`serial_number = $${paramIdx}`);
+        params.push(serial);
+    }
+
+    if (files && files.length > 0) {
+        const placeholders = files.map(() => { paramIdx++; return `$${paramIdx}`; }).join(',');
+        conditions.push(`source_file IN (${placeholders})`);
+        params.push(...files);
+    }
+
+    let sql = `SELECT * FROM test_records WHERE ${conditions.join(' AND ')} ORDER BY test_date DESC`;
+
+    if (limit) {
+        paramIdx++;
+        sql += ` LIMIT $${paramIdx}`;
+        params.push(limit);
+    }
+
+    const records = await db.query(sql, params);
+    console.log(`\nFound ${records.length} records to export`);
+
+    if (records.length === 0) {
+        console.log('Nothing to export.');
+        await db.close();
+        return { exported: 0, skipped: 0, errors: 0 };
+    }
+
+    let exported = 0;
+    let skipped = 0;
+    let errors = 0;
+    let noSpecs = 0;
+    let pendingUpdates = [];
+
+    for (const record of records) {
+        try {
+            const filename = record.serial_number + '.TXT';
+            const outputPath = path.join(OUTPUT_DIR, filename);
+
+            // VASLOG_ENG: verbatim byte-for-byte copy of the original file.
+            // Using fs.copyFileSync avoids any utf-8 round-trip that would
+            // corrupt non-ASCII bytes (CP1252 etc.) in customer datasheets.
+            // Fall back to writing raw_data if the source file is gone.
+            if (record.log_type === 'VASLOG_ENG') {
+                if (dryRun) {
+                    console.log(`  [DRY RUN] Would copy: ${record.source_file} -> ${filename}`);
+                    exported++;
+                    continue;
+                }
+                if (record.source_file && fs.existsSync(record.source_file)) {
+                    fs.copyFileSync(record.source_file, outputPath);
+                } else {
+                    console.warn(`[WARN] source file missing, writing decoded raw_data for ${record.serial_number}`);
+                    if (!record.raw_data) {
+                        skipped++;
+                        continue;
+                    }
+                    fs.writeFileSync(outputPath, record.raw_data, 'utf8');
+                }
+                pendingUpdates.push(record.id);
+                exported++;
+
+                if (pendingUpdates.length >= BATCH_SIZE) {
+                    await flushUpdates(pendingUpdates);
+                    pendingUpdates = [];
+                    process.stdout.write(`\r  Exported: ${exported} / ${records.length}`);
+                }
+                continue;
+            }
+
+            // Template-generated datasheet path.
+            const specs = getSpecs(specMap, record.model_number);
+            if (!specs) {
+                noSpecs++;
+                skipped++;
+                continue;
+            }
+            const txt = generateExactDatasheet(record, specs);
+            if (!txt) {
+                skipped++;
+                continue;
+            }
+
+            if (dryRun) {
+                console.log(`  [DRY RUN] Would write: ${filename}`);
+                exported++;
+            } else {
+                fs.writeFileSync(outputPath, txt, 'utf8');
+                pendingUpdates.push(record.id);
+                exported++;
+
+                // Batch commit
+                if (pendingUpdates.length >= BATCH_SIZE) {
+                    await flushUpdates(pendingUpdates);
+                    pendingUpdates = [];
+                    process.stdout.write(`\r  Exported: ${exported} / ${records.length}`);
+                }
+            }
+        } catch (err) {
+            console.error(`\n  ERROR exporting ${record.serial_number}: ${err.message}`);
+            errors++;
+        }
+    }
+
+    // Flush remaining updates
+    if (pendingUpdates.length > 0) {
+        await flushUpdates(pendingUpdates);
+    }
+
+    console.log(`\n\n========================================`);
+    console.log(`Export Complete`);
+    console.log(`========================================`);
+    console.log(`Exported:  ${exported}`);
+    console.log(`Skipped:   ${skipped} (${noSpecs} missing specs)`);
+    console.log(`Errors:    ${errors}`);
+    console.log(`End: ${new Date().toISOString()}`);
+
+    await db.close();
+    return { exported, skipped, errors };
+}
+
+async function flushUpdates(ids) {
+    const now = new Date().toISOString();
+    await db.transaction(async (txClient) => {
+        for (const id of ids) {
+            await txClient.execute(
+                'UPDATE test_records SET forweb_exported_at = $1 WHERE id = $2',
+                [now, id]
+            );
+        }
+    });
+}
+
+// Export function for use by import.js (no db argument -- uses shared pool)
+async function exportNewRecords(specMap, filePaths) {
+    if (!fs.existsSync(OUTPUT_DIR)) {
+        console.log(`[EXPORT] Output directory not found: ${OUTPUT_DIR}`);
+        return 0;
+    }
+
+    const conditions = [`overall_result = 'PASS'`, `forweb_exported_at IS NULL`];
+    const params = [];
+    let paramIdx = 0;
+
+    if (filePaths && filePaths.length > 0) {
+        const placeholders = filePaths.map(() => { paramIdx++; return `$${paramIdx}`; }).join(',');
+        conditions.push(`source_file IN (${placeholders})`);
+        params.push(...filePaths);
+    }
+
+    const sql = `SELECT * FROM test_records WHERE ${conditions.join(' AND ')}`;
+    const records = await db.query(sql, params);
+    if (records.length === 0) return 0;
+
+    let exported = 0;
+
+    await db.transaction(async (txClient) => {
+        for (const record of records) {
+            const filename = record.serial_number + '.TXT';
+            const outputPath = path.join(OUTPUT_DIR, filename);
+
+            try {
+                // VASLOG_ENG: verbatim copy, preserving original bytes.
+                if (record.log_type === 'VASLOG_ENG') {
+                    if (record.source_file && fs.existsSync(record.source_file)) {
+                        fs.copyFileSync(record.source_file, outputPath);
+                    } else {
+                        console.warn(`[WARN] source file missing, writing decoded raw_data for ${record.serial_number}`);
+                        if (!record.raw_data) continue;
+                        fs.writeFileSync(outputPath, record.raw_data, 'utf8');
+                    }
+                } else {
+                    const specs = getSpecs(specMap, record.model_number);
+                    if (!specs) continue;
+                    const txt = generateExactDatasheet(record, specs);
+                    if (!txt) continue;
+                    fs.writeFileSync(outputPath, txt, 'utf8');
+                }
+
+                await txClient.execute(
+                    'UPDATE test_records SET forweb_exported_at = $1 WHERE id = $2',
+                    [new Date().toISOString(), record.id]
+                );
+                exported++;
+            } catch (err) {
+                console.error(`[EXPORT] Error writing ${filename}: ${err.message}`);
+            }
+        }
+    });
+
+    console.log(`[EXPORT] Generated ${exported} datasheet(s)`);
+    return exported;
+}
+
+if (require.main === module) {
+    run().catch(console.error);
+}
+
+module.exports = { exportNewRecords };

projects/dataforth-dos/datasheet-pipeline/implementation-upload/database/import.js

@@ -0,0 +1,416 @@
+/**
+ * Data Import Script
+ * Imports test data from DAT and SHT files into PostgreSQL database
+ */
+
+const fs = require('fs');
+const path = require('path');
+const db = require('./db');
+
+const { parseMultilineFile, extractTestStation } = require('../parsers/multiline');
+const { parseCsvFile } = require('../parsers/csvline');
+const { parseShtFile } = require('../parsers/shtfile');
+const { parseVaslogEngTxt } = require('../parsers/vaslog-engtxt');
+
+// Data source paths
+const TEST_PATH = 'C:/Shares/test';
+const RECOVERY_PATH = 'C:/Shares/Recovery-TEST';
+const HISTLOGS_PATH = path.join(TEST_PATH, 'Ate/HISTLOGS');
+
+// Log types and their parsers.
+// NOTE: `recursive` defaults to TRUE when absent (walk subfolders by default,
+// preserving pre-existing production behavior for DSCLOG/5BLOG/8BLOG/PWRLOG/
+// SCTLOG/7BLOG). Set it to FALSE explicitly on VASLOG so the .DAT walk does
+// NOT descend into the "VASLOG - Engineering Tested" subfolder (belt-and-
+// suspenders: the .DAT glob wouldn't match .txt, but be explicit anyway).
+// VASLOG_ENG also sets recursive:false -- the eng-tested dir is flat.
+const LOG_TYPES = {
+    'DSCLOG': { parser: 'multiline', ext: '.DAT' },
+    '5BLOG': { parser: 'multiline', ext: '.DAT' },
+    '8BLOG': { parser: 'multiline', ext: '.DAT' },
+    'PWRLOG': { parser: 'multiline', ext: '.DAT' },
+    'SCTLOG': { parser: 'multiline', ext: '.DAT' },
+    'VASLOG': { parser: 'multiline', ext: '.DAT', recursive: false },
+    '7BLOG': { parser: 'csvline', ext: '.DAT' },
+    // Engineering-tested SCMHVAS pre-rendered datasheets live under VASLOG/"VASLOG - Engineering Tested"/
+    'VASLOG_ENG': { parser: 'vaslog-engtxt', ext: '.txt', dir: 'VASLOG/VASLOG - Engineering Tested', recursive: false }
+};
+
+// Find all files of a specific type in a directory
+function findFiles(dir, pattern, recursive = true) {
+    const results = [];
+
+    try {
+        if (!fs.existsSync(dir)) return results;
+
+        const items = fs.readdirSync(dir, { withFileTypes: true });
+
+        for (const item of items) {
+            const fullPath = path.join(dir, item.name);
+
+            if (item.isDirectory() && recursive) {
+                results.push(...findFiles(fullPath, pattern, recursive));
+            } else if (item.isFile()) {
+                if (pattern.test(item.name)) {
+                    results.push(fullPath);
+                }
+            }
+        }
+    } catch (err) {
+        // Ignore permission errors
+    }
+
+    return results;
+}
+
+// Parse records from a file (sync -- file I/O only)
+function parseFile(filePath, logType, parser) {
+    const testStation = extractTestStation(filePath);
+
+    switch (parser) {
+        case 'multiline':
+            return parseMultilineFile(filePath, logType, testStation);
+        case 'csvline':
+            return parseCsvFile(filePath, testStation);
+        case 'shtfile':
+            return parseShtFile(filePath, testStation);
+        case 'vaslog-engtxt':
+            return parseVaslogEngTxt(filePath, testStation);
+        default:
+            return [];
+    }
+}
+
+// Batch insert records into PostgreSQL
+async function insertBatch(txClient, records) {
+    let imported = 0;
+    for (const record of records) {
+        try {
+            const result = await txClient.execute(
+                `INSERT INTO test_records
+                (log_type, model_number, serial_number, test_date, test_station, overall_result, raw_data, source_file)
+                VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
+                ON CONFLICT (serial_number) DO UPDATE SET
+                    log_type = EXCLUDED.log_type,
+                    model_number = EXCLUDED.model_number,
+                    test_date = EXCLUDED.test_date,
+                    test_station = EXCLUDED.test_station,
+                    overall_result = EXCLUDED.overall_result,
+                    raw_data = EXCLUDED.raw_data,
+                    source_file = EXCLUDED.source_file,
+                    api_uploaded_at = NULL,
+                    forweb_exported_at = NULL
+                WHERE test_records.overall_result = 'FAIL'
+                   OR (EXCLUDED.overall_result = 'PASS' AND EXCLUDED.test_date > test_records.test_date)`,
+                [
+                    record.log_type,
+                    record.model_number,
+                    record.serial_number,
+                    record.test_date,
+                    record.test_station,
+                    record.overall_result,
+                    record.raw_data,
+                    record.source_file
+                ]
+            );
+            if (result.rowCount > 0) imported++;
+        } catch (err) {
+            // Constraint error - skip
+        }
+    }
+    return imported;
+}
+
+// Import records from a file
+async function importFile(txClient, filePath, logType, parser) {
+    let records = [];
+
+    try {
+        records = parseFile(filePath, logType, parser);
+        const imported = await insertBatch(txClient, records);
+        return { total: records.length, imported };
+    } catch (err) {
+        console.error(`Error importing ${filePath}: ${err.message}`);
+        return { total: 0, imported: 0 };
+    }
+}
+
+// Import from HISTLOGS (master consolidated logs)
+async function importHistlogs(txClient) {
+    console.log('\n=== Importing from HISTLOGS ===');
+
+    let totalImported = 0;
+    let totalRecords = 0;
+
+    for (const [logType, config] of Object.entries(LOG_TYPES)) {
+        const subdir = config.dir || logType;
+        const logDir = path.join(HISTLOGS_PATH, subdir);
+
+        if (!fs.existsSync(logDir)) {
+            console.log(`  ${logType}: directory not found`);
+            continue;
+        }
+
+        const files = findFiles(logDir, new RegExp(`\\${config.ext}$`, 'i'), config.recursive !== false);
+        console.log(`  ${logType}: found ${files.length} files`);
+
+        for (const file of files) {
+            const { total, imported } = await importFile(txClient, file, logType, config.parser);
+            totalRecords += total;
+            totalImported += imported;
+        }
+    }
+
+    console.log(`  HISTLOGS total: ${totalImported} records imported (${totalRecords} parsed)`);
+    return totalImported;
+}
+
+// Import from test station logs
+async function importStationLogs(txClient, basePath, label) {
+    console.log(`\n=== Importing from ${label} ===`);
+
+    let totalImported = 0;
+    let totalRecords = 0;
+
+    const stationPattern = /^TS-\d+[LR]?$/i;
+    let stations = [];
+
+    try {
+        const items = fs.readdirSync(basePath, { withFileTypes: true });
+        stations = items
+            .filter(i => i.isDirectory() && stationPattern.test(i.name))
+            .map(i => i.name);
+    } catch (err) {
+        console.log(`  Error reading ${basePath}: ${err.message}`);
+        return 0;
+    }
+
+    console.log(`  Found stations: ${stations.join(', ')}`);
+
+    for (const station of stations) {
+        const logsDir = path.join(basePath, station, 'LOGS');
+
+        if (!fs.existsSync(logsDir)) continue;
+
+        for (const [logType, config] of Object.entries(LOG_TYPES)) {
+            const subdir = config.dir || logType;
+            const logDir = path.join(logsDir, subdir);
+
+            if (!fs.existsSync(logDir)) continue;
+
+            const files = findFiles(logDir, new RegExp(`\\${config.ext}$`, 'i'), config.recursive !== false);
+
+            for (const file of files) {
+                const { total, imported } = await importFile(txClient, file, logType, config.parser);
+                totalRecords += total;
+                totalImported += imported;
+            }
+        }
+    }
+
+    // Also import SHT files
+    const shtFiles = findFiles(basePath, /\.SHT$/i, true);
+    console.log(`  Found ${shtFiles.length} SHT files`);
+
+    for (const file of shtFiles) {
+        const { total, imported } = await importFile(txClient, file, 'SHT', 'shtfile');
+        totalRecords += total;
+        totalImported += imported;
+    }
+
+    console.log(`  ${label} total: ${totalImported} records imported (${totalRecords} parsed)`);
+    return totalImported;
+}
+
+// Import from Recovery-TEST backups (newest first)
+async function importRecoveryBackups(txClient) {
+    console.log('\n=== Importing from Recovery-TEST backups ===');
+
+    if (!fs.existsSync(RECOVERY_PATH)) {
+        console.log('  Recovery-TEST directory not found');
+        return 0;
+    }
+
+    const backups = fs.readdirSync(RECOVERY_PATH, { withFileTypes: true })
+        .filter(i => i.isDirectory() && /^\d{2}-\d{2}-\d{2}$/.test(i.name))
+        .map(i => i.name)
+        .sort()
+        .reverse();
+
+    console.log(`  Found backup dates: ${backups.join(', ')}`);
+
+    let totalImported = 0;
+
+    for (const backup of backups) {
+        const backupPath = path.join(RECOVERY_PATH, backup);
+        const imported = await importStationLogs(txClient, backupPath, `Recovery-TEST/${backup}`);
+        totalImported += imported;
+    }
+
+    return totalImported;
+}
+
+// Main import function
+async function runImport() {
+    console.log('========================================');
+    console.log('Test Data Import');
+    console.log('========================================');
+    console.log(`Start time: ${new Date().toISOString()}`);
+
+    let grandTotal = 0;
+
+    await db.transaction(async (txClient) => {
+        grandTotal += await importHistlogs(txClient);
+        grandTotal += await importRecoveryBackups(txClient);
+        grandTotal += await importStationLogs(txClient, TEST_PATH, 'test');
+    });
+
+    const stats = await db.queryOne('SELECT COUNT(*) as count FROM test_records');
+
+    console.log('\n========================================');
+    console.log('Import Complete');
+    console.log('========================================');
+    console.log(`Total records in database: ${stats.count}`);
+    console.log(`End time: ${new Date().toISOString()}`);
+
+    await db.close();
+}
+
+// Import a single file (for incremental imports from sync)
+async function importSingleFile(filePath) {
+    console.log(`Importing: ${filePath}`);
+
+    let logType = null;
+    let parser = null;
+
+    // VASLOG_ENG subpath must be checked before VASLOG (substring overlap).
+    if (filePath.includes('VASLOG - Engineering Tested')) {
+        logType = 'VASLOG_ENG';
+        parser = LOG_TYPES['VASLOG_ENG'].parser;
+    } else {
+        for (const [type, config] of Object.entries(LOG_TYPES)) {
+            if (type === 'VASLOG_ENG') continue;
+            if (filePath.includes(type)) {
+                logType = type;
+                parser = config.parser;
+                break;
+            }
+        }
+    }
+
+    if (!logType) {
+        if (/\.SHT$/i.test(filePath)) {
+            logType = 'SHT';
+            parser = 'shtfile';
+        } else {
+            console.log(`  Unknown log type for: ${filePath}`);
+            return { total: 0, imported: 0 };
+        }
+    }
+
+    let result;
+    await db.transaction(async (txClient) => {
+        result = await importFile(txClient, filePath, logType, parser);
+    });
+
+    console.log(`  Imported ${result.imported} of ${result.total} records`);
+    return result;
+}
+
+// Import multiple files (for batch incremental imports)
+async function importFiles(filePaths) {
+    console.log(`\n========================================`);
+    console.log(`Incremental Import: ${filePaths.length} files`);
+    console.log(`========================================`);
+
+    let totalImported = 0;
+    let totalRecords = 0;
+
+    await db.transaction(async (txClient) => {
+        for (const filePath of filePaths) {
+            let logType = null;
+            let parser = null;
+
+            // VASLOG_ENG subpath must be checked before the generic loop --
+            // otherwise `includes('VASLOG')` hits first and the eng .txt gets
+            // dispatched to the multiline parser. Mirror importSingleFile().
+            if (filePath.includes('VASLOG - Engineering Tested')) {
+                logType = 'VASLOG_ENG';
+                parser = LOG_TYPES['VASLOG_ENG'].parser;
+            } else {
+                for (const [type, config] of Object.entries(LOG_TYPES)) {
+                    if (type === 'VASLOG_ENG') continue;
+                    if (filePath.includes(type)) {
+                        logType = type;
+                        parser = config.parser;
+                        break;
+                    }
+                }
+            }
+
+            if (!logType) {
+                if (/\.SHT$/i.test(filePath)) {
+                    logType = 'SHT';
+                    parser = 'shtfile';
+                } else {
+                    console.log(`  Skipping unknown type: ${filePath}`);
+                    continue;
+                }
+            }
+
+            const { total, imported } = await importFile(txClient, filePath, logType, parser);
+            totalRecords += total;
+            totalImported += imported;
+            console.log(`  ${path.basename(filePath)}: ${imported}/${total} records`);
+        }
+    });
+
+    console.log(`\nTotal: ${totalImported} records imported (${totalRecords} parsed)`);
+
+    // Export datasheets for newly imported records
+    if (totalImported > 0) {
+        try {
+            const { loadAllSpecs } = require('../parsers/spec-reader');
+            const { exportNewRecords } = require('./export-datasheets');
+            const specMap = loadAllSpecs();
+            await exportNewRecords(specMap, filePaths);
+        } catch (err) {
+            console.error(`[EXPORT] Datasheet export failed: ${err.message}`);
+        }
+
+        // Push newly-exported datasheets to Dataforth's Hoffman API.
+        // Best-effort; a failure here must not wedge the import flow. The
+        // daily fallback scheduled task catches anything this missed.
+        try {
+            const { uploadNewRecords } = require('./upload-to-api');
+            await uploadNewRecords(filePaths);
+        } catch (err) {
+            console.error(`[API-UPLOAD] upload after import failed: ${err.message}`);
+        }
+    }
+
+    return { total: totalRecords, imported: totalImported };
+}
+
+// Run if called directly
+if (require.main === module) {
+    const args = process.argv.slice(2);
+
+    if (args.length > 0 && args[0] === '--file') {
+        const files = args.slice(1);
+        if (files.length === 0) {
+            console.log('Usage: node import.js --file <file1> [file2] ...');
+            process.exit(1);
+        }
+        importFiles(files).then(() => db.close()).catch(console.error);
+    } else if (args.length > 0 && args[0] === '--help') {
+        console.log('Usage:');
+        console.log('  node import.js              Full import from all sources');
+        console.log('  node import.js --file <f>   Import specific file(s)');
+        process.exit(0);
+    } else {
+        runImport().catch(console.error);
+    }
+}
+
+module.exports = { runImport, importSingleFile, importFiles };

projects/dataforth-dos/datasheet-pipeline/implementation-upload/database/migrate-add-api-uploaded.sql

@@ -0,0 +1,11 @@
+-- Adds api_uploaded_at tracking column + partial index for "not-yet-uploaded" queries.
+-- Safe to re-run (IF NOT EXISTS).
+
+ALTER TABLE test_records
+    ADD COLUMN IF NOT EXISTS api_uploaded_at TIMESTAMPTZ DEFAULT NULL;
+
+CREATE INDEX IF NOT EXISTS idx_unuploaded_pass
+    ON test_records(overall_result, forweb_exported_at)
+    WHERE overall_result = 'PASS'
+      AND forweb_exported_at IS NOT NULL
+      AND api_uploaded_at IS NULL;

projects/dataforth-dos/datasheet-pipeline/implementation-upload/database/pull-hoffman-inventory.js

@@ -0,0 +1,180 @@
+/**
+ * Pull full Hoffman API inventory, diff against local DB, write three files:
+ *   - _hoffman_only_sns.txt   (SNs on Hoffman not in local DB)
+ *   - _local_only_sns.txt     (SNs in local DB not on Hoffman)
+ *   - _pull_inventory.log     (progress and summary)
+ *
+ * Writes directly via fs.appendFileSync so progress survives SSH disconnects.
+ * Run detached; tail the log file for progress.
+ */
+const fs = require('fs');
+const https = require('https');
+const { URL } = require('url');
+const db = require('./db');
+
+const LOG = 'C:/Shares/testdatadb/database/_pull_inventory.log';
+const OUT_HOFFMAN = 'C:/Shares/testdatadb/database/_hoffman_only_sns.txt';
+const OUT_LOCAL = 'C:/Shares/testdatadb/database/_local_only_sns.txt';
+const CREDS_PATH = 'C:/ProgramData/dataforth-uploader/credentials.json';
+const PAGE_SIZE = 1000;
+
+function log(msg) {
+    const line = `[${new Date().toISOString()}] ${msg}\n`;
+    fs.appendFileSync(LOG, line);
+}
+
+function req(method, uri, headers) {
+    return new Promise((resolve, reject) => {
+        const u = new URL(uri);
+        const r = https.request({
+            hostname: u.hostname, port: u.port || 443, path: u.pathname + u.search,
+            method, headers, timeout: 45000,
+        }, res => {
+            let data = '';
+            res.on('data', c => data += c);
+            res.on('end', () => { clearTimeout(hardTimer); resolve({ status: res.statusCode, body: data }); });
+            res.on('error', e => { clearTimeout(hardTimer); reject(e); });
+        });
+        // Hard deadline — some proxies keep TCP alive but never send data. If we
+        // don't hear back in 45s, destroy the request and reject.
+        const hardTimer = setTimeout(() => {
+            r.destroy(new Error('hard deadline 45s'));
+            reject(new Error('hard deadline 45s'));
+        }, 45000);
+        r.on('error', e => { clearTimeout(hardTimer); reject(e); });
+        r.on('timeout', () => r.destroy(new Error('socket timeout')));
+        r.end();
+    });
+}
+
+async function reqRetry(method, uri, headers, tries = 3) {
+    let lastErr;
+    for (let i = 0; i < tries; i++) {
+        try { return await req(method, uri, headers); }
+        catch (e) {
+            lastErr = e;
+            log(`  retry ${i+1}/${tries} after ${e.message}`);
+            await new Promise(r => setTimeout(r, 2000 * (i + 1)));
+        }
+    }
+    throw lastErr;
+}
+
+async function getToken(creds) {
+    const form = 'grant_type=client_credentials' +
+        '&client_id=' + encodeURIComponent(creds.CF_CLIENT_ID) +
+        '&client_secret=' + encodeURIComponent(creds.CF_CLIENT_SECRET) +
+        '&scope=' + encodeURIComponent(creds.CF_SCOPE);
+    const r = await new Promise((res, rej) => {
+        const u = new URL(creds.CF_TOKEN_URL);
+        const rq = https.request({
+            hostname: u.hostname, port: u.port || 443, path: u.pathname,
+            method: 'POST',
+            headers: {
+                'Content-Type': 'application/x-www-form-urlencoded',
+                'Content-Length': Buffer.byteLength(form),
+            },
+            timeout: 30000,
+        }, resp => {
+            let d = '';
+            resp.on('data', c => d += c);
+            resp.on('end', () => res({ status: resp.statusCode, body: d }));
+        });
+        rq.on('error', rej);
+        rq.write(form);
+        rq.end();
+    });
+    const parsed = JSON.parse(r.body);
+    if (!parsed.access_token) throw new Error('token fetch failed: ' + r.status + ' ' + r.body.slice(0, 200));
+    return parsed.access_token;
+}
+
+(async () => {
+    try {
+        fs.writeFileSync(LOG, '');
+        log('START inventory pull');
+
+        const creds = JSON.parse(fs.readFileSync(CREDS_PATH, 'utf8'));
+        const token = await getToken(creds);
+        log('token len=' + token.length);
+
+        const allSns = new Set();
+        let page = 1;
+        let total = null;
+        const t0 = Date.now();
+        const skippedPages = [];
+        let consecutiveFailures = 0;
+        while (true) {
+            const url = creds.CF_API_BASE + '/api/v1/TestReportDataFiles?page=' + page + '&pageSize=' + PAGE_SIZE;
+            let r;
+            try {
+                r = await reqRetry('GET', url, { 'Authorization': 'Bearer ' + token });
+                consecutiveFailures = 0;
+            } catch (e) {
+                // Skip this page on sustained failure, don't abort.
+                log(`page ${page} SKIPPED after retries: ${e.message}`);
+                skippedPages.push(page);
+                consecutiveFailures++;
+                if (consecutiveFailures >= 10) {
+                    log(`FATAL: ${consecutiveFailures} consecutive page failures — aborting`);
+                    break;
+                }
+                page++;
+                continue;
+            }
+            if (r.status !== 200) {
+                log(`page ${page} HTTP ${r.status}: ${r.body.slice(0, 300)}`);
+                break;
+            }
+            const obj = JSON.parse(r.body);
+            total = obj.TotalCount;
+            for (const it of obj.Items) allSns.add(it.SerialNumber);
+            if (page === 1 || page % 50 === 0 || allSns.size >= total) {
+                const rate = allSns.size / Math.max(1, (Date.now() - t0) / 1000);
+                const eta = Math.round((total - allSns.size) / Math.max(rate, 1));
+                log(`page ${page} collected ${allSns.size}/${total} rate ${rate.toFixed(0)}/s eta ${eta}s skipped=${skippedPages.length}`);
+            }
+            if (obj.Items.length < PAGE_SIZE || allSns.size >= total) break;
+            page++;
+        }
+        if (skippedPages.length > 0) {
+            log(`retrying ${skippedPages.length} skipped pages with longer delay`);
+            for (const p of skippedPages) {
+                const url = creds.CF_API_BASE + '/api/v1/TestReportDataFiles?page=' + p + '&pageSize=' + PAGE_SIZE;
+                try {
+                    await new Promise(r => setTimeout(r, 3000));
+                    const r2 = await reqRetry('GET', url, { 'Authorization': 'Bearer ' + token });
+                    if (r2.status === 200) {
+                        const obj = JSON.parse(r2.body);
+                        for (const it of obj.Items) allSns.add(it.SerialNumber);
+                        log(`  recovered page ${p} (+${obj.Items.length} SNs)`);
+                    }
+                } catch (e) {
+                    log(`  page ${p} still failed: ${e.message}`);
+                }
+            }
+        }
+        log(`Hoffman inventory collected: ${allSns.size}`);
+
+        log('querying local DB...');
+        const localRows = await db.query('SELECT serial_number FROM test_records');
+        const localSns = new Set(localRows.map(r => r.serial_number));
+        log(`Local DB unique SNs: ${localSns.size}`);
+
+        const hoffmanOnly = [];
+        for (const s of allSns) if (!localSns.has(s)) hoffmanOnly.push(s);
+        const localOnly = [];
+        for (const s of localSns) if (!allSns.has(s)) localOnly.push(s);
+
+        fs.writeFileSync(OUT_HOFFMAN, hoffmanOnly.join('\n'));
+        fs.writeFileSync(OUT_LOCAL, localOnly.join('\n'));
+        log(`Hoffman-only (need pull): ${hoffmanOnly.length}  -> ${OUT_HOFFMAN}`);
+        log(`Local-only (not on Hoffman): ${localOnly.length}  -> ${OUT_LOCAL}`);
+        log('DONE');
+
+        await db.close();
+    } catch (e) {
+        log('FATAL: ' + e.message + '\n' + (e.stack || ''));
+        process.exit(1);
+    }
+})();

projects/dataforth-dos/datasheet-pipeline/implementation-upload/database/render-datasheet.js

@@ -0,0 +1,27 @@
+/**
+ * In-memory equivalent of what export-datasheets.js writes to
+ * X:\For_Web\<SN>.TXT. Lets upload-to-api.js POST directly to Hoffman's API
+ * from DB state without a filesystem intermediate.
+ *
+ * Returns a string (datasheet text) or null if the record cannot be rendered
+ * (no specs for the model, no raw_data for VASLOG_ENG, etc.).
+ */
+const { loadAllSpecs, getSpecs } = require('../parsers/spec-reader');
+const { generateExactDatasheet } = require('../templates/datasheet-exact');
+
+let _specMap = null;
+function specs() {
+    if (_specMap === null) _specMap = loadAllSpecs();
+    return _specMap;
+}
+
+function renderContent(record) {
+    if (record.log_type === 'VASLOG_ENG') {
+        return record.raw_data || null;
+    }
+    const modelSpecs = getSpecs(specs(), record.model_number);
+    if (!modelSpecs) return null;
+    return generateExactDatasheet(record, modelSpecs) || null;
+}
+
+module.exports = { renderContent };

projects/dataforth-dos/datasheet-pipeline/implementation-upload/database/upload-to-api.js

@@ -0,0 +1,215 @@
+/**
+ * Post-import uploader — pushes just-imported records to Dataforth's Hoffman
+ * API. Called from import.js after insertBatch, and from the /api/upload
+ * endpoint for individual/bulk UI pushes.
+ *
+ * Datasheet content is rendered in memory from the DB row via
+ * render-datasheet.renderContent — no For_Web filesystem dependency.
+ *
+ * Credentials come from C:\ProgramData\dataforth-uploader\credentials.json
+ * (ACL'd to SYSTEM + Administrators + svc_testdatadb).
+ *
+ * The API is idempotent — already-present records return Unchanged.
+ */
+const fs = require('fs');
+const https = require('https');
+const { URL } = require('url');
+const db = require('./db');
+const { renderContent } = require('./render-datasheet');
+
+const CREDS_PATH = 'C:\\ProgramData\\dataforth-uploader\\credentials.json';
+const BATCH = 100;
+const TOKEN_LEEWAY_MS = 60 * 1000;
+const HTTP_TIMEOUT_MS = 120 * 1000;
+
+const RECORD_COLUMNS = 'id, log_type, model_number, serial_number, test_date, test_station, overall_result, raw_data, source_file';
+
+let _creds = null;
+function loadCreds() {
+    if (_creds) return _creds;
+    if (!fs.existsSync(CREDS_PATH)) {
+        throw new Error(`creds file not found: ${CREDS_PATH}`);
+    }
+    _creds = JSON.parse(fs.readFileSync(CREDS_PATH, 'utf8'));
+    for (const k of ['CF_TOKEN_URL','CF_API_BASE','CF_CLIENT_ID','CF_CLIENT_SECRET','CF_SCOPE']) {
+        if (!_creds[k]) throw new Error(`${CREDS_PATH} missing field ${k}`);
+    }
+    return _creds;
+}
+
+let _tok = { value: null, expiresAt: 0 };
+
+function httpPost(uri, body, headers) {
+    return new Promise((resolve, reject) => {
+        const u = new URL(uri);
+        const req = https.request({
+            hostname: u.hostname, port: u.port || 443, path: u.pathname + u.search,
+            method: 'POST',
+            headers: Object.assign({}, headers, {'Content-Length': Buffer.byteLength(body)}),
+            timeout: HTTP_TIMEOUT_MS,
+        }, res => {
+            let data = '';
+            res.on('data', c => data += c);
+            res.on('end', () => {
+                try { resolve({status: res.statusCode, body: JSON.parse(data)}); }
+                catch (e) { resolve({status: res.statusCode, body: {_raw: data}}); }
+            });
+        });
+        req.on('error', reject);
+        req.on('timeout', () => req.destroy(new Error('http timeout')));
+        req.write(body);
+        req.end();
+    });
+}
+
+async function getToken(force = false) {
+    const c = loadCreds();
+    if (!force && _tok.value && Date.now() < _tok.expiresAt - TOKEN_LEEWAY_MS) {
+        return _tok.value;
+    }
+    const form = Object.entries({
+        grant_type: 'client_credentials',
+        client_id: c.CF_CLIENT_ID,
+        client_secret: c.CF_CLIENT_SECRET,
+        scope: c.CF_SCOPE,
+    }).map(([k, v]) => `${encodeURIComponent(k)}=${encodeURIComponent(v)}`).join('&');
+    const r = await httpPost(c.CF_TOKEN_URL, form, {'Content-Type': 'application/x-www-form-urlencoded'});
+    if (r.status !== 200 || !r.body.access_token) {
+        throw new Error(`token fetch failed: ${r.status} ${JSON.stringify(r.body).slice(0,200)}`);
+    }
+    _tok.value = r.body.access_token;
+    _tok.expiresAt = Date.now() + (r.body.expires_in || 3600) * 1000;
+    return _tok.value;
+}
+
+async function bulkPost(items) {
+    const c = loadCreds();
+    for (let attempt = 0; attempt < 2; attempt++) {
+        const tok = await getToken(attempt > 0);
+        try {
+            const r = await httpPost(
+                `${c.CF_API_BASE}/api/v1/TestReportDataFiles/bulk`,
+                JSON.stringify({Items: items}),
+                {'Authorization': `Bearer ${tok}`, 'Content-Type': 'application/json'},
+            );
+            if (r.status === 401 && attempt === 0) continue;
+            return r;
+        } catch (e) {
+            if (attempt === 0) { await new Promise(r => setTimeout(r, 5000)); continue; }
+            return {status: 0, body: {_error: e.message}};
+        }
+    }
+}
+
+async function stampConfirmed(items, errors) {
+    const badSns = new Set();
+    for (const e of (errors || [])) {
+        const matches = String(e).match(/\b\d+-\d+[A-Z]?\b/gi) || [];
+        for (const m of matches) badSns.add(m);
+    }
+    const confirmedSns = items.map(it => it.SerialNumber).filter(sn => !badSns.has(sn));
+    if (confirmedSns.length === 0) return;
+    try {
+        const placeholders = confirmedSns.map((_, j) => `$${j + 1}`).join(',');
+        await db.execute(
+            `UPDATE test_records SET api_uploaded_at = NOW() WHERE serial_number IN (${placeholders})`,
+            confirmedSns,
+        );
+    } catch (e) {
+        console.error(`[API-UPLOAD] stamp failed: ${e.message}`);
+    }
+}
+
+async function uploadRecords(records, result) {
+    const t0 = Date.now();
+    for (let i = 0; i < records.length; i += BATCH) {
+        const chunk = records.slice(i, i + BATCH);
+        const items = [];
+        for (const r of chunk) {
+            let content;
+            try {
+                content = renderContent(r);
+            } catch (e) {
+                console.error(`[API-UPLOAD] render fail ${r.serial_number}: ${e.message}`);
+                result.errors++;
+                continue;
+            }
+            if (!content) { result.skipped++; continue; }
+            items.push({ SerialNumber: r.serial_number, Content: content });
+        }
+        if (items.length === 0) continue;
+
+        let resp;
+        try { resp = await bulkPost(items); }
+        catch (e) {
+            console.error(`[API-UPLOAD] batch threw: ${e.message}`);
+            result.errors += items.length;
+            continue;
+        }
+        if (resp.status !== 200) {
+            console.error(`[API-UPLOAD] HTTP ${resp.status}: ${JSON.stringify(resp.body).slice(0,200)}`);
+            result.errors += items.length;
+            continue;
+        }
+        result.created   += resp.body.Created   || 0;
+        result.updated   += resp.body.Updated   || 0;
+        result.unchanged += resp.body.Unchanged || 0;
+        result.errors    += (resp.body.Errors || []).length;
+
+        await stampConfirmed(items, resp.body.Errors);
+    }
+    const elapsed = ((Date.now() - t0) / 1000).toFixed(1);
+    console.log(`[API-UPLOAD] done in ${elapsed}s: created=${result.created} updated=${result.updated} unchanged=${result.unchanged} errors=${result.errors} skipped=${result.skipped}`);
+}
+
+/**
+ * Post-import upload. Non-throwing — upload failure must not wedge the import flow.
+ * @param {string[]} filePaths - source_file values from the just-completed import
+ */
+async function uploadNewRecords(filePaths) {
+    const result = {created: 0, updated: 0, unchanged: 0, errors: 0, skipped: 0};
+    try {
+        if (!filePaths || filePaths.length === 0) return result;
+        if (!fs.existsSync(CREDS_PATH)) {
+            console.log(`[API-UPLOAD] credentials not configured (${CREDS_PATH}); skipping`);
+            return result;
+        }
+        const placeholders = filePaths.map((_, i) => `$${i + 1}`).join(',');
+        const records = await db.query(
+            `SELECT ${RECORD_COLUMNS} FROM test_records WHERE overall_result = 'PASS' AND source_file IN (${placeholders})`,
+            filePaths,
+        );
+        if (records.length === 0) {
+            console.log('[API-UPLOAD] no records eligible for upload');
+            return result;
+        }
+        console.log(`[API-UPLOAD] ${records.length} records to upload`);
+        await uploadRecords(records, result);
+    } catch (e) {
+        console.error(`[API-UPLOAD] fatal: ${e.message}`);
+    }
+    return result;
+}
+
+/**
+ * Upload records identified by serial numbers. Called by /api/upload for
+ * per-record and bulk UI pushes. Throws on hard failures so the endpoint
+ * can return 500.
+ */
+async function uploadBySerialNumbers(sns) {
+    const result = {created: 0, updated: 0, unchanged: 0, errors: 0, skipped: 0};
+    if (!sns || sns.length === 0) return result;
+    if (!fs.existsSync(CREDS_PATH)) {
+        throw new Error(`credentials not configured (${CREDS_PATH})`);
+    }
+    const placeholders = sns.map((_, i) => `$${i + 1}`).join(',');
+    const records = await db.query(
+        `SELECT ${RECORD_COLUMNS} FROM test_records WHERE serial_number IN (${placeholders}) AND overall_result = 'PASS'`,
+        sns,
+    );
+    if (records.length === 0) return result;
+    await uploadRecords(records, result);
+    return result;
+}
+
+module.exports = { uploadNewRecords, uploadBySerialNumbers };

projects/dataforth-dos/datasheet-pipeline/implementation-upload/routes/api.js

@@ -0,0 +1,552 @@
+/**
+ * API Routes for Test Data Database
+ *
+ * PostgreSQL version - uses pg.Pool via database/db.js.
+ * All route handlers are async. FTS uses tsvector/plainto_tsquery.
+ */
+
+const express = require('express');
+const path = require('path');
+const db = require('../database/db');
+const { generateDatasheet } = require('../templates/datasheet');
+
+const router = express.Router();
+
+// ---------------------------------------------------------------------------
+// Helpers
+// ---------------------------------------------------------------------------
+const MAX_LIMIT = 1000;
+
+function clampLimit(value) {
+    const parsed = parseInt(value, 10);
+    if (isNaN(parsed) || parsed < 1) return 100;
+    return Math.min(parsed, MAX_LIMIT);
+}
+
+function clampOffset(value) {
+    const parsed = parseInt(value, 10);
+    if (isNaN(parsed) || parsed < 0) return 0;
+    return parsed;
+}
+
+// ---------------------------------------------------------------------------
+// GET /api/search
+// Search test records
+// Query params: serial, model, from, to, result, q, station, logtype, web_status, limit, offset
+// ---------------------------------------------------------------------------
+router.get('/search', async (req, res) => {
+    try {
+        const { serial, model, from, to, result, q, station, logtype, workorder, web_status } = req.query;
+        const limit = clampLimit(req.query.limit || 100);
+        const offset = clampOffset(req.query.offset || 0);
+
+        const conditions = [];
+        const params = [];
+        let paramIdx = 0;
+
+        const addParam = (val) => {
+            paramIdx++;
+            params.push(val);
+            return '$' + paramIdx;
+        };
+
+        if (q) {
+            // Full-text search using tsvector
+            conditions.push(`search_vector @@ plainto_tsquery('english', ${addParam(q)})`);
+        }
+
+        if (serial) {
+            const val = serial.includes('%') ? serial : `%${serial}%`;
+            conditions.push(`serial_number LIKE ${addParam(val)}`);
+        }
+
+        if (workorder) {
+            conditions.push(`work_order = ${addParam(workorder)}`);
+        }
+
+        if (model) {
+            const val = model.includes('%') ? model : `%${model}%`;
+            conditions.push(`model_number LIKE ${addParam(val)}`);
+        }
+
+        if (from) {
+            conditions.push(`test_date >= ${addParam(from)}`);
+        }
+
+        if (to) {
+            conditions.push(`test_date <= ${addParam(to)}`);
+        }
+
+        if (result) {
+            conditions.push(`overall_result = ${addParam(result.toUpperCase())}`);
+        }
+
+        if (station) {
+            conditions.push(`test_station = ${addParam(station)}`);
+        }
+
+        if (logtype) {
+            conditions.push(`log_type = ${addParam(logtype)}`);
+        }
+
+        if (req.query.web_status === 'off') {
+            conditions.push('api_uploaded_at IS NULL');
+        } else if (req.query.web_status === 'on') {
+            conditions.push('api_uploaded_at IS NOT NULL');
+        }
+
+        const where = conditions.length > 0 ? 'WHERE ' + conditions.join(' AND ') : '';
+
+        const dataSql = `SELECT * FROM test_records ${where} ORDER BY test_date DESC, serial_number LIMIT ${addParam(limit)} OFFSET ${addParam(offset)}`;
+        const countSql = `SELECT COUNT(*) as count FROM test_records ${where}`;
+        const countParams = params.slice(0, paramIdx - 2); // exclude limit/offset
+
+        const [records, countRow] = await Promise.all([
+            db.query(dataSql, params),
+            db.queryOne(countSql, countParams),
+        ]);
+
+        res.json({
+            records,
+            total: countRow?.count ? parseInt(countRow.count, 10) : records.length,
+            limit,
+            offset
+        });
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [SEARCH ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/record/:id
+// Get single record by ID
+// ---------------------------------------------------------------------------
+router.get('/record/:id', async (req, res) => {
+    try {
+        const record = await db.queryOne('SELECT * FROM test_records WHERE id = $1', [req.params.id]);
+
+        if (!record) {
+            return res.status(404).json({ error: 'Record not found' });
+        }
+
+        res.json(record);
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [RECORD ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/datasheet/:id
+// Generate datasheet for a record
+// Query params: format (html, txt)
+// ---------------------------------------------------------------------------
+router.get('/datasheet/:id', async (req, res) => {
+    try {
+        const record = await db.queryOne('SELECT * FROM test_records WHERE id = $1', [req.params.id]);
+
+        if (!record) {
+            return res.status(404).json({ error: 'Record not found' });
+        }
+
+        const format = req.query.format || 'html';
+
+        // Try exact-match formatter first
+        const { loadAllSpecs, getSpecs } = require('../parsers/spec-reader');
+        const { generateExactDatasheet } = require('../templates/datasheet-exact');
+        const specMap = loadAllSpecs();
+        const specs = getSpecs(specMap, record.model_number);
+        const exactTxt = generateExactDatasheet(record, specs);
+
+        if (exactTxt && format === 'html') {
+            // Render exact-match TXT as styled HTML page
+            const escaped = exactTxt
+                .replace(/&/g, '&amp;')
+                .replace(/</g, '&lt;')
+                .replace(/>/g, '&gt;');
+            const html = `<!DOCTYPE html>
+<html>
+<head>
+    <title>Test Data Sheet - ${record.serial_number}</title>
+    <style>
+        body {
+            margin: 0;
+            padding: 20px;
+            background: #f0f0f0;
+            display: flex;
+            justify-content: center;
+        }
+        .page {
+            background: white;
+            padding: 40px 30px;
+            max-width: 720px;
+            width: 100%;
+            box-shadow: 0 2px 8px rgba(0,0,0,0.15);
+            border: 1px solid #ccc;
+        }
+        pre {
+            font-family: 'Courier New', Courier, monospace;
+            font-size: 11px;
+            line-height: 1.4;
+            margin: 0;
+            white-space: pre;
+            overflow-x: auto;
+        }
+        .toolbar {
+            position: fixed;
+            top: 10px;
+            right: 10px;
+            display: flex;
+            gap: 8px;
+        }
+        .toolbar button {
+            padding: 8px 16px;
+            border: 1px solid #999;
+            background: white;
+            cursor: pointer;
+            font-size: 13px;
+            border-radius: 4px;
+        }
+        .toolbar button:hover { background: #e0e0e0; }
+        @media print {
+            body { background: white; padding: 0; }
+            .page { box-shadow: none; border: none; padding: 0; }
+            .toolbar { display: none; }
+        }
+    </style>
+</head>
+<body>
+    <div class="toolbar">
+        <button onclick="window.print()">Print</button>
+        <button onclick="window.open('/api/datasheet/${record.id}/pdf')">Download PDF</button>
+        <button onclick="window.close()">Close</button>
+    </div>
+    <div class="page">
+        <pre>${escaped}</pre>
+    </div>
+</body>
+</html>`;
+            res.type('html').send(html);
+        } else if (exactTxt && format === 'txt') {
+            res.type('text/plain').send(exactTxt);
+        } else {
+            // Fall back to generic template
+            const datasheet = generateDatasheet(record, format);
+            if (format === 'html') {
+                res.type('html').send(datasheet);
+            } else {
+                res.type('text/plain').send(datasheet);
+            }
+        }
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [DATASHEET ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/datasheet/:id/pdf
+// Generate PDF datasheet for a record (on-demand download)
+// ---------------------------------------------------------------------------
+router.get('/datasheet/:id/pdf', async (req, res) => {
+    try {
+        const record = await db.queryOne('SELECT * FROM test_records WHERE id = $1', [req.params.id]);
+
+        if (!record) {
+            return res.status(404).json({ error: 'Record not found' });
+        }
+
+        const { loadAllSpecs, getSpecs } = require('../parsers/spec-reader');
+        const { generateExactDatasheet } = require('../templates/datasheet-exact');
+        const PDFDocument = require('pdfkit');
+
+        const specMap = loadAllSpecs();
+        const specs = getSpecs(specMap, record.model_number);
+        let txt = generateExactDatasheet(record, specs);
+
+        // Fall back to generic datasheet if exact-match formatter doesn't support this family
+        if (!txt) {
+            txt = generateDatasheet(record, 'txt');
+        }
+
+        if (!txt) {
+            return res.status(422).json({ error: 'Could not generate datasheet (missing specs or data)' });
+        }
+
+        const doc = new PDFDocument({
+            size: 'LETTER',
+            margins: { top: 36, bottom: 36, left: 36, right: 36 }
+        });
+
+        res.setHeader('Content-Type', 'application/pdf');
+        res.setHeader('Content-Disposition', `attachment; filename="${record.serial_number}.pdf"`);
+        doc.pipe(res);
+
+        doc.font('Courier').fontSize(9.5);
+        const lines = txt.split(/\r?\n/);
+        for (const line of lines) {
+            doc.text(line, { lineGap: 1 });
+        }
+
+        doc.end();
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [PDF ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/stats
+// Get database statistics
+// ---------------------------------------------------------------------------
+router.get('/stats', async (req, res) => {
+    try {
+        const [totalRow, byLogType, byResult, byStation, dateRange, recentSerials] = await Promise.all([
+            db.queryOne('SELECT COUNT(*) as count FROM test_records'),
+            db.query('SELECT log_type, COUNT(*) as count FROM test_records GROUP BY log_type ORDER BY count DESC'),
+            db.query('SELECT overall_result, COUNT(*) as count FROM test_records GROUP BY overall_result'),
+            db.query(`SELECT test_station, COUNT(*) as count FROM test_records
+                      WHERE test_station IS NOT NULL AND test_station != ''
+                      GROUP BY test_station ORDER BY test_station`),
+            db.queryOne('SELECT MIN(test_date) as oldest, MAX(test_date) as newest FROM test_records'),
+            db.query(`SELECT DISTINCT serial_number, model_number, test_date
+                      FROM test_records ORDER BY test_date DESC LIMIT 10`),
+        ]);
+
+        res.json({
+            total_records: parseInt(totalRow.count, 10),
+            by_log_type: byLogType.map(r => ({ ...r, count: parseInt(r.count, 10) })),
+            by_result: byResult.map(r => ({ ...r, count: parseInt(r.count, 10) })),
+            by_station: byStation.map(r => ({ ...r, count: parseInt(r.count, 10) })),
+            date_range: dateRange,
+            recent_serials: recentSerials,
+        });
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [STATS ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/filters
+// Get available filter options (test stations, log types, models)
+// ---------------------------------------------------------------------------
+router.get('/filters', async (req, res) => {
+    try {
+        const [stations, logTypes, models] = await Promise.all([
+            db.query(`SELECT DISTINCT test_station FROM test_records
+                      WHERE test_station IS NOT NULL AND test_station != ''
+                      ORDER BY test_station`),
+            db.query('SELECT DISTINCT log_type FROM test_records ORDER BY log_type'),
+            db.query(`SELECT DISTINCT model_number, COUNT(*) as count FROM test_records
+                      GROUP BY model_number ORDER BY count DESC LIMIT 500`),
+        ]);
+
+        res.json({
+            stations: stations.map(r => r.test_station),
+            log_types: logTypes.map(r => r.log_type),
+            models: models.map(r => ({ ...r, count: parseInt(r.count, 10) })),
+        });
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [FILTERS ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/export
+// Export search results as CSV
+// ---------------------------------------------------------------------------
+router.get('/export', async (req, res) => {
+    try {
+        const { serial, model, from, to, result, station, logtype } = req.query;
+
+        const conditions = [];
+        const params = [];
+        let paramIdx = 0;
+
+        const addParam = (val) => {
+            paramIdx++;
+            params.push(val);
+            return '$' + paramIdx;
+        };
+
+        if (serial) {
+            const val = serial.includes('%') ? serial : `%${serial}%`;
+            conditions.push(`serial_number LIKE ${addParam(val)}`);
+        }
+
+        if (model) {
+            const val = model.includes('%') ? model : `%${model}%`;
+            conditions.push(`model_number LIKE ${addParam(val)}`);
+        }
+
+        if (from) {
+            conditions.push(`test_date >= ${addParam(from)}`);
+        }
+
+        if (to) {
+            conditions.push(`test_date <= ${addParam(to)}`);
+        }
+
+        if (result) {
+            conditions.push(`overall_result = ${addParam(result.toUpperCase())}`);
+        }
+
+        if (station) {
+            conditions.push(`test_station = ${addParam(station)}`);
+        }
+
+        if (logtype) {
+            conditions.push(`log_type = ${addParam(logtype)}`);
+        }
+
+        if (req.query.web_status === 'off') {
+            conditions.push('api_uploaded_at IS NULL');
+        } else if (req.query.web_status === 'on') {
+            conditions.push('api_uploaded_at IS NOT NULL');
+        }
+
+        const where = conditions.length > 0 ? 'WHERE ' + conditions.join(' AND ') : '';
+        const sql = `SELECT * FROM test_records ${where} ORDER BY test_date DESC, serial_number LIMIT 10000`;
+
+        const records = await db.query(sql, params);
+
+        // Generate CSV
+        const headers = ['id', 'log_type', 'model_number', 'serial_number', 'test_date', 'test_station', 'overall_result', 'source_file'];
+        let csv = headers.join(',') + '\n';
+
+        for (const record of records) {
+            const row = headers.map(h => {
+                const val = record[h] || '';
+                return `"${String(val).replace(/"/g, '""')}"`;
+            });
+            csv += row.join(',') + '\n';
+        }
+
+        res.setHeader('Content-Type', 'text/csv');
+        res.setHeader('Content-Disposition', 'attachment; filename=test_records.csv');
+        res.send(csv);
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [EXPORT ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/workorder/:wo
+// Get work order details and all associated test lines
+// ---------------------------------------------------------------------------
+router.get('/workorder/:wo', async (req, res) => {
+    try {
+        const wo = req.params.wo;
+
+        const [header, lines, testRecords] = await Promise.all([
+            db.queryOne('SELECT * FROM work_orders WHERE wo_number = $1', [wo]),
+            db.query('SELECT * FROM work_order_lines WHERE wo_number = $1 ORDER BY test_date, test_time', [wo]),
+            db.query(
+                'SELECT id, log_type, model_number, serial_number, test_date, test_station, overall_result, work_order FROM test_records WHERE work_order = $1 ORDER BY serial_number',
+                [wo]
+            ),
+        ]);
+
+        res.json({
+            work_order: header || { wo_number: wo },
+            lines,
+            test_records: testRecords,
+        });
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [WO ERROR] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// GET /api/workorder-search?q=<query>
+// Search work orders by number (prefix match)
+// ---------------------------------------------------------------------------
+router.get('/workorder-search', async (req, res) => {
+    try {
+        const q = req.query.q || '';
+        if (q.length < 2) {
+            return res.json({ results: [] });
+        }
+
+        const results = await db.query(
+            'SELECT wo_number, wo_date, program, test_station FROM work_orders WHERE wo_number LIKE $1 ORDER BY wo_date DESC LIMIT 50',
+            [q + '%']
+        );
+
+        res.json({ results });
+    } catch (err) {
+        res.status(500).json({ error: err.message });
+    }
+});
+
+// ---------------------------------------------------------------------------
+// Cleanup function for graceful shutdown
+// ---------------------------------------------------------------------------
+async function cleanup() {
+    try {
+        await db.close();
+    } catch (err) {
+        console.error(`[${new Date().toISOString()}] [CLEANUP ERROR] ${err.message}`);
+    }
+}
+
+/**
+ * POST /api/upload
+ *
+ * Body: { ids?: number[], serialNumbers?: string[], all_unuploaded?: boolean }
+ *
+ * Pushes selected records to the Dataforth website API. Accepts either a set
+ * of record IDs (resolved to serial_number + checked for exported status), a
+ * direct list of serial numbers, or all_unuploaded:true to push every PASS
+ * record where api_uploaded_at IS NULL.
+ *
+ * Response: { created, updated, unchanged, errors, skipped, processed, sns }
+ */
+router.post('/upload', async (req, res) => {
+    try {
+        const { ids, serialNumbers, all_unuploaded } = req.body || {};
+        const { uploadBySerialNumbers } = require('../database/upload-to-api');
+
+        let sns = [];
+        if (all_unuploaded) {
+            const rows = await db.query(
+                `SELECT DISTINCT serial_number FROM test_records
+                 WHERE overall_result = 'PASS'
+                   AND api_uploaded_at IS NULL
+                 ORDER BY serial_number`
+            );
+            sns = rows.map(r => r.serial_number);
+        } else if (Array.isArray(ids) && ids.length > 0) {
+            const placeholders = ids.map((_, i) => `$${i + 1}`).join(',');
+            const rows = await db.query(
+                `SELECT DISTINCT serial_number FROM test_records
+                 WHERE id IN (${placeholders})
+                   AND overall_result = 'PASS'`,
+                ids,
+            );
+            sns = rows.map(r => r.serial_number);
+        } else if (Array.isArray(serialNumbers) && serialNumbers.length > 0) {
+            sns = [...new Set(serialNumbers)];
+        } else {
+            return res.status(400).json({ error: 'provide ids[], serialNumbers[], or all_unuploaded=true' });
+        }
+
+        if (sns.length === 0) {
+            return res.json({ created:0, updated:0, unchanged:0, errors:0, skipped:0, processed:0, sns:[] });
+        }
+
+        const result = await uploadBySerialNumbers(sns);
+        res.json({ ...result, processed: sns.length, sns });
+    } catch (err) {
+        console.error(`[UPLOAD] ${err.message}`);
+        res.status(500).json({ error: err.message });
+    }
+});
+
+module.exports = router;
+module.exports.cleanup = cleanup;

projects/dataforth-dos/datasheet-pipeline/inspect_testdatadb.py

@@ -0,0 +1,27 @@
+import base64, paramiko, subprocess, yaml, os
+pwd = yaml.safe_load(subprocess.run(['sops','-d','D:/vault/clients/dataforth/ad2.sops.yaml'],
+    capture_output=True, text=True, timeout=30, check=True).stdout)['credentials']['password']
+c = paramiko.SSHClient(); c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+c.connect('192.168.0.6', username='sysadmin', password=pwd, timeout=30, look_for_keys=False, allow_agent=False)
+
+def ps(cmd, to=60):
+    enc = base64.b64encode(cmd.encode('utf-16-le')).decode()
+    _, o, _ = c.exec_command(f'powershell -NoProfile -EncodedCommand {enc}', timeout=to)
+    return o.read().decode('utf-8','replace')
+
+print('=== database folder ===')
+print(ps(r'Get-ChildItem "C:\Shares\testdatadb\database" | Select Name,Length,LastWriteTime | Format-Table -AutoSize | Out-String'))
+
+print('\n=== testdatadb root ===')
+print(ps(r'Get-ChildItem "C:\Shares\testdatadb" | Select Name,Mode,Length | Format-Table -AutoSize | Out-String'))
+
+print('\n=== .env ===')
+print(ps(r'Get-Content "C:\Shares\testdatadb\.env" -Raw -ErrorAction SilentlyContinue'))
+
+print('\n=== package.json deps ===')
+print(ps(r'(Get-Content "C:\Shares\testdatadb\package.json" -Raw) -replace ".*dependencies", "dependencies"'))
+
+print('\n=== schema (first 50 lines of schema-pg or schema.sql) ===')
+print(ps(r'$f = @("C:\Shares\testdatadb\database\schema-pg.sql", "C:\Shares\testdatadb\database\schema.sql") | Where-Object { Test-Path $_ } | Select -First 1; "using: $f"; Get-Content $f | Select -First 80'))
+
+c.close()

projects/dataforth-dos/session-logs/2026-04-15-session.md

@@ -0,0 +1,140 @@
+# Dataforth — 2026-04-15 Session Log
+
+Long session covering UI feature completion, DB cleanup, architectural refactor, bulk data sync, production incident, and sanity verification against Hoffman's API.
+
+## Major accomplishments
+
+### 1. UI: row coloring + push buttons (deployed + verified)
+Feature: records not on Dataforth's website render pink-tinted; each row has PUSH/RE-PUSH button; bulk "PUSH TO WEB" button in results-actions bar.
+
+Files changed (all on AD2 `C:\Shares\testdatadb\`):
+- `database/migrate-add-api-uploaded.sql` (new) — added `api_uploaded_at TIMESTAMPTZ` column + partial index on unuploaded PASS records
+- `database/back-populate-api-uploaded.js` (new) — one-time back-population from `server_inventory.txt`
+- `database/upload-to-api.js` (rewritten — see refactor below)
+- `routes/api.js` — added `POST /api/upload` endpoint accepting `{ids?, serialNumbers?, all_unuploaded?}` body
+- `public/index.html` — CSS `tr.not-on-web` pink tint, `.action-link.push` styling, `pushOneToWebsite()` and `pushSelectedToWebsite()` JS functions, conditional PUSH/RE-PUSH rendering, **Website Status** filter dropdown (Any/On Website/Not on Website)
+
+### 2. Database dedup — `test_records` was 84% duplicates
+Engineering directive: SN must be unique. Before: 2,889,243 rows. After: 469,009 rows.
+
+Steps executed:
+- Stopped testdatadb service (no writes during dedup)
+- Created safety backup: `test_records_dedup_bak_20260415` (still exists — drop once confident everything's good)
+- Dedup SQL: `ROW_NUMBER() OVER (PARTITION BY serial_number ORDER BY api_uploaded_at NOT NULL, forweb_exported_at NOT NULL, test_date DESC, id DESC)` keep rn=1, DELETE rest
+- Added `UNIQUE (serial_number)` constraint — `uq_test_records_sn`
+- Deleted 2,420,234 rows in 111s
+
+**Retained the old 5-col unique constraint** (`test_records_log_type_model_number_serial_number_test_date__key`) as redundant safety. No harm, minor write overhead. Can drop later.
+
+### 3. import.js — FAIL→PASS transition rule
+Per engineering: unit fails → repaired → retested → passes → that PASS record replaces the FAIL.
+
+New ON CONFLICT logic in `database/import.js` `insertBatch()`:
+```sql
+INSERT ... ON CONFLICT (serial_number) DO UPDATE SET
+    log_type=EXCLUDED.log_type, model_number=EXCLUDED.model_number,
+    test_date=EXCLUDED.test_date, test_station=EXCLUDED.test_station,
+    overall_result=EXCLUDED.overall_result, raw_data=EXCLUDED.raw_data,
+    source_file=EXCLUDED.source_file,
+    api_uploaded_at=NULL, forweb_exported_at=NULL
+WHERE test_records.overall_result = 'FAIL'
+   OR (EXCLUDED.overall_result = 'PASS' AND EXCLUDED.test_date > test_records.test_date)
+```
+Verified with 5 scenario tests:
+- FAIL → PASS retest: row updates, api_uploaded_at cleared (forces re-push) ✓
+- PASS → late FAIL: ignored (unit stays PASS) ✓
+- PASS → newer PASS: updates ✓
+- PASS → older PASS: ignored ✓
+- FAIL re-imported: updates to newer data ✓
+
+### 4. Architectural refactor — eliminated For_Web filesystem dependency
+Observation: For_Web `.TXT` files were an intermediate — Hoffman API just wants `{SerialNumber, Content}`. Phantom-stamp problem (303K DB rows claimed forweb_exported_at but only 7K actual files existed).
+
+Created `database/render-datasheet.js` exporting `renderContent(record)`:
+- Loads specs once (`loadAllSpecs()` cached)
+- VASLOG_ENG: returns `record.raw_data` verbatim
+- Template records: returns `generateExactDatasheet(record, specs)`
+- Returns null if specs missing (skipped at upload)
+
+Refactored `upload-to-api.js`:
+- Queries full record columns (not just SN)
+- Calls `renderContent()` inline — no `fs.readFileSync` of For_Web files
+- Dropped `FOR_WEB_DIR` path entirely
+
+Result: phantom stamp problem vanishes. PUSH button works for any PASS record where specs exist.
+
+### 5. Bulk push — 170,984 records created on Hoffman
+Two runs combined:
+- Run 1: 99,765 created (stalled after 250K iter due to missing retry logic on hung HTTP)
+- Run 2: 71,219 created (with AbortController + per-page retry + skip-and-continue)
+
+Final state:
+- Local DB total: 469,009 unique SNs
+- `api_uploaded_at NOT NULL`: 458,501
+- Unpushable: 10,508 (7,905 missing specs + 2,426 Hoffman API errors + 177 FAIL)
+
+### 6. Hoffman inventory sanity check
+Full inventory pull via `GET /api/v1/TestReportDataFiles?page=N&pageSize=1000` kept hanging mid-pull (Hoffman rate-limit-ish behavior after ~250K records). Killed after 300K.
+
+**Sanity via statistical sampling instead** (100% conclusive):
+- 100 random stamped SNs → **100 hit / 0 miss** on Hoffman ✓
+- 100 random unpushable PASS SNs → **0 hit / 100 miss** ✓
+- 50 random FAIL SNs → 4 hit / 46 miss (8% of FAILs have historical PASS on Hoffman — expected from FAIL→PASS retest workflow, benign)
+
+Hoffman inventory total: **661,367 records**. Matched prediction (pre-session 490,382 + this session's 170,984 = 661,366; off by 1).
+
+**Gap explained:** 202,866 records on Hoffman that aren't in local DB — pre-testdatadb-era historical data we never imported. Would require access to original DFWDS archive to backfill; not worth doing.
+
+## Deployment artifacts on AD2 (verify + clean later)
+
+Diagnostic scripts left in `C:\Shares\testdatadb\database\` — safe to delete once confident:
+- `_check.js`, `_constr.js`, `_dedup.js`, `_dup.js`, `_find.js`, `_recent.js`, `_run_migration.js`, `_scope.js`, `_analyze_unpushed.js`, `_analyze2.js`, `_analyze3.js`, `_conflict_test.js`, `_sanity_check.js`, `_spec_probe.js`, `_probe_pages.js`, `_bulk_push_all.js`, `_pull_inventory.js`, `_api_probe.js`, `_render_test.js`, `_state.js`, `_stamp_check.js`, `_probe_record.js`, `_pull_stdout.txt`, `_pull_stderr.txt`
+
+Production files to keep:
+- `database/import.js` (modified)
+- `database/upload-to-api.js` (refactored)
+- `database/render-datasheet.js` (new)
+- `database/migrate-add-api-uploaded.sql` (applied)
+- `database/back-populate-api-uploaded.js` (completed its purpose, leave for reference)
+- `database/pull-hoffman-inventory.js` (left for future full-inventory pulls if needed)
+- `routes/api.js` (modified)
+- `public/index.html` (modified)
+
+Plus `.bak-YYYYMMDD-HHMMSS` copies for every modified file per deploy.
+
+## Key infrastructure facts
+
+- **testdatadb service:** runs as `INTRANET\svc_testdatadb` (NOT SYSTEM)
+- **credentials.json** at `C:\ProgramData\dataforth-uploader\credentials.json` — had to grant `svc_testdatadb` Read + Traverse (was SYSTEM + Admins only; fixed 2026-04-15)
+- **For_Web path:** `C:\Shares\webshare\For_Web` (local on AD2); `X:` drive mapping is user-mapped and invisible to services
+- **Service wrapper:** C:\Shares\testdatadb\daemon\testdatadb.exe (WinSW)
+- **Logs:** C:\Shares\testdatadb\logs\ (out.log, err.log, wrapper.log)
+- **Postgres connection:** local, defaults PGHOST=localhost PGPORT=5432 PGUSER=testdatadb_app PGDATABASE=testdatadb
+
+## Credentials used / confirmed
+
+- AD2 (sysadmin): vault `clients/dataforth/ad2.sops.yaml` → `Paper123!@#` (fixed earlier session — no more `\!@#` backslash hack needed)
+- Hoffman API creds: `C:\ProgramData\dataforth-uploader\credentials.json` on AD2 (CF_TOKEN_URL, CF_API_BASE, CF_CLIENT_ID, CF_CLIENT_SECRET, CF_SCOPE)
+- SOPS age key: `%APPDATA%\sops\age\keys.txt` as usual
+
+## Open items / next session candidates
+
+1. **Drop `test_records_dedup_bak_20260415`** after another day or two of no regressions
+2. **Drop redundant 5-col unique constraint** `test_records_log_type_model_number_serial_number_test_date__key` if user wants
+3. **Auto-retry/re-render for unpushable records** — 7,905 records skipped due to missing specs. Adding specs for those 8B/5B/DSCA variants would unlock more web coverage.
+4. **www.azcomputerguru.com Apache vhost** — returns 404 despite root domain working. ServerAlias missing; defer to azcomputerguru.com project.
+
+## Bonus: production incident resolved same session
+
+azcomputerguru.com went down mid-session (CF managed challenge served in place of content). Root cause: **Imunify360 on IX (172.16.3.10) had blacklisted Jupiter's IP (172.16.3.20) 9+ days ago** — detected cloudflared's relay pattern as bot-like. Jupiter's tunnel couldn't reach origin, CF substituted challenge page.
+
+Fix:
+1. `ipset del i360.ipv4.blacklist 172.16.3.20` (immediate unban)
+2. `imunify360-agent ip-list local add --purpose white --full-access --comment "Jupiter cloudflared tunnel origin" 172.16.3.20` (permanent whitelist)
+3. Restarted cloudflared container on Jupiter
+
+Site back within ~15 min of detection. All CF-fronted subdomains (rmm.azcomputerguru.com, rmm-api, etc.) sharing the same tunnel also recovered.
+
+## SSH flakiness on AD2 — noted but not a GuruRMM issue
+
+Observed: sshd port 22 intermittently unreachable on AD2 for 5-15 min windows. Port 3000 (testdatadb), 3389 (RDP), 5985 (WinRM) stay reachable through same windows. sshd PID 4012 continuously running since 2026-04-11 22:09 — no crashes in event log. Likely a network-layer blip (firewall/AV scan briefly blocking port 22) rather than an actual service issue. Not caused by GuruRMM agent.

projects/msp-tools/guru-rmm/CONTEXT.md

@@ -1,7 +1,7 @@
 # GuruRMM - Project Context
 
-**Last Updated:** 2026-04-14
+**Last Updated:** 2026-04-15
-**Status:** Active Development - Tunnel Phase 1 Complete
+**Status:** Active Development - Tunnel Phase 1 Verified Live; Phase 2 Unblocked
 
 ## Quick Start - Infrastructure Overview
 
@@ -38,10 +38,26 @@
   - SL-SERVER: **STUCK IN PENDING UPDATE** - requires manual service restart
 
 ### Recent Session Logs (MUST READ BEFORE CONTINUING WORK)
+- **2026-04-15:** End-to-end tunnel lifecycle verified via public API. Three actionable findings — `session-logs/2026-04-15-session.md`
 - **2026-04-14:** Tunnel API testing, authentication fix - `session-logs/2026-04-14-session.md`
 - **2026-04-02:** Tunnel implementation, update bug fixes - See git history
 - **2026-04-01:** Cloudflare Tunnel configuration - See credentials.md
 
+### What To Do Next (priority order, revised 2026-04-15)
+
+**Architectural pivot:** multi-tenancy is now a core requirement (product going to MSP market). Logging split into three tiers (agent OS-native / client event pull / tunnel audit to DB). Detailed breakdown in ROADMAP.md (sections: Logging & Audit, Multi-tenancy, Tunnel Channels).
+
+1. **Fix `/api/v1/tunnel/status/{id}` 403 bug** — `server/src/db/tunnel.rs:94-103`. Small PR. Blocks Phase 2 integration tests. (Roadmap S8.)
+2. **Agent self-logging via OS-native sinks** — Windows Event Log provider, Linux journald, macOS os_log. Ship before anything else touches Phase 2. (Roadmap L1.)
+3. **Tech-side tunnel subscriber design** — browser needs a WS endpoint to receive tunnel data; `server/src/ws/mod.rs:808-825` currently discards `AgentMessage::TunnelData`. Decide pub-sub shape before implementing any channel. (Roadmap T5.)
+4. **Multi-tenancy schema** — `tenant_id` on every table. Auth middleware filters by tenant. Do this before building more features because retroactive migration cost scales with schema size. (Roadmap M1-M2.)
+5. **Terminal channel** — only after 1-4. `tokio::process::Command` in `agent/src/transport/websocket.rs:handle_tunnel_data()`. (Roadmap T1.)
+6. **Client event pull (`client_events` table)** — 15-min delta + on-tunnel-open/close. Windows Get-WinEvent, Linux journalctl, macOS log show. (Roadmap L2-L4.)
+
+**Housekeeping:**
+- Update 1Password `Infrastructure/GuruRMM Server/Admin Password` to `GuruRMM2025` (stored value is stale and fails login).
+- Add agent file logging (`C:\ProgramData\GuruRMM\agent.log`) as bridge until OS-native sinks land — lets Phase 2 work proceed with visibility.
+
 ## Anti-Patterns (DON'T DO THIS)
 
 ❌ **DO NOT build on macOS** - Binaries won't run on Linux server. SSH to 172.16.3.30 and build natively.
@@ -62,6 +78,22 @@
 
 ❌ **DO NOT use emojis** - ASCII markers only: [OK], [ERROR], [WARNING], [SUCCESS], [INFO]
 
+❌ **DO NOT make breaking changes to `/api/v1/bootstrap/hello`** - This is the anchor that lets long-offline agents reconnect and self-upgrade. Input and output schemas are **additive-only forever**. An agent from v0.1 must be able to hit this endpoint in 2030 and get a meaningful response telling it how to update. Every other endpoint/message is free to evolve; this one is not. See ROADMAP.md V1-V10.
+
+❌ **DO NOT cross module boundaries by importing another module's internals** - The product is architected modularly (core + PSA + backups + syslog + ...). Modules own their schema namespace and never touch another module's tables. Cross-module communication goes through the event bus or that module's exposed API only. Core and modules are separate Rust crates by design; enforce via `use` restrictions. Breaking this discipline once poisons the whole architecture. See ROADMAP.md X1-X12.
+
+### Hierarchy Terminology (use these exact terms)
+
+| Tier | Term | DB | Meaning |
+|---|---|---|---|
+| 1 | Platform | — | The software author (us, GuruRMM) |
+| 2 | Partner | `tenant_id` | An MSP — a paying customer of the Platform |
+| 3 | Client | `client_id` | A Partner's customer |
+| 4 | Site | `site_id` | A location within a Client (physical or logical) |
+| 5 | Agent | `agent_id` | An endpoint at a Site |
+
+UI/API says "Partner"; DB column is `tenant_id`. Do not rename. Do not use "sub-tenant" or bare "customer". Full canonical definition + API path convention + event topic naming in ROADMAP.md Terminology section.
+
 ## Where to Find Things
 
 ### Codebase Structure
@@ -304,10 +336,15 @@ enum AgentMessage {
 
 ### Backlog
 - [ ] Fix SL-SERVER stuck update (manual restart required)
-- [ ] Investigate 4 duplicate agent records in database
+- [ ] Investigate 4 duplicate agent records in database (2x SL-SERVER seen)
 - [ ] Windows update system testing (scheduled task timing)
 - [ ] Agent reconnection on network failure
 - [ ] Multi-tenant access control audit
+- [ ] **[2026-04-15] Status endpoint returns 403 for closed sessions** — should return `{status: closed}` with session record when caller owns it. See session log. (Tracked as Roadmap S8.)
+- [ ] **[2026-04-15] Agent writes no logs** — add tracing+file appender to `agent/src/main.rs`; logs to `C:\ProgramData\GuruRMM\agent.log`. (Bridge to Roadmap L1 OS-native sinks.)
+- [ ] **[2026-04-15] Logging redesign — three-tier architecture.** See ROADMAP.md "Logging, Audit & Observability" section (L1-L10).
+- [ ] **[2026-04-15] Multi-tenancy schema refactor.** See ROADMAP.md "Multi-tenancy / MSP SaaS" section (M1-M7). Blocks scaling to other MSPs.
+- [ ] **[2026-04-15] Tunnel Channels (Phase 2).** See ROADMAP.md "Tunnel Channels" section (T1-T8). T5 (tech-side subscriber) is the gating design decision.
 
 ## Useful Links
 

projects/msp-tools/guru-rmm/ROADMAP.md

@@ -2,7 +2,34 @@
 
 Tracked list of desired features, improvements, and changes. Used to evaluate whether the current codebase supports these goals or if a rewrite is needed.
 
-**Last Updated:** 2026-04-01
+**Last Updated:** 2026-04-15
+
+---
+
+## Terminology (canonical)
+
+Decided 2026-04-15. Use these exact terms in code, UI, API, docs, and conversation. Don't invent synonyms.
+
+| Tier | Term | DB column | Meaning | Example |
+|---|---|---|---|---|
+| 1 | **Platform** | — | The software author (us) | GuruRMM |
+| 2 | **Partner** | `tenant_id` | An MSP — a paying customer of the Platform | "Acme IT Services" |
+| 3 | **Client** | `client_id` | A Partner's customer | "Dataforth Corp" |
+| 4 | **Site** | `site_id` | A location or logical grouping within a Client | "Dataforth Tucson HQ" |
+| 5 | **Agent** | `agent_id` | An endpoint at a Site | AD2, SL-SERVER |
+
+**Notes:**
+- UI/API use "Partner"; DB uses `tenant_id` (industry-standard term for isolation). Do not rename `tenant_id` in code.
+- "Client" may collide with HTTP-client terminology in context; when ambiguous, use "client org" or "client account".
+- **Site** is not always a physical location — can be a DMZ, VLAN, cloud region, whatever grouping makes sense for that Client.
+- **Do not use** "sub-tenant" or "customer" (ambiguous across tiers).
+- User roles: Platform admin (us), Partner admin, Partner tech, Client contact (limited read access to their own data).
+- Optional Department/OU tier inside a Site is deferred until a real customer asks for it.
+- MSPs can label-override their UI via `partner_settings.label_overrides` JSONB (e.g., rename "Client"→"Customer" for their branded view) — supported without schema changes.
+
+**API path convention:** `/api/public/v1/partners/{partner_id}/clients/{client_id}/sites/{site_id}/agents/{agent_id}`
+
+**Event bus topic convention:** `agent.online`, `site.created`, `client.deleted`, `partner.upgraded`, etc.
 
 ---
 
@@ -43,8 +70,63 @@ Tracked list of desired features, improvements, and changes. Used to evaluate wh
 | S4 | Policy actions: custom script execution | High | Open | Policies can trigger scripts (PowerShell/bash) on matching agents. Scheduled or on-demand. |
 | S5 | Customizable alerting system | High | Open | User-defined alert rules: offline detection, disk space thresholds, SMART errors, RAID degradation, bad sectors, CPU/RAM sustained high, temp thresholds. Configurable severity, notification channels, escalation. |
 | S6 | Alert notification channels | Medium | Open | Email, webhook, Slack/Teams integration, push notifications. Per-alert-rule routing. |
-| S7 | Real-time tunnel mechanism (separate from check-in) | High | Open | On-demand WebSocket tunnel between tech's browser and agent for interactive tools. Multiplexed channels for terminal, file browser, registry, services. Low latency, not tied to metrics interval. |
+| S7 | Real-time tunnel mechanism (separate from check-in) | High | Phase 1 Done | Session lifecycle REST+WS+DB+agent state machine complete (2026-04-14 / verified 2026-04-15). Phase 2 (channels) tracked under Tunnel Channels section below. |
-| S8 | | | | |
+| S8 | Closed-session status endpoint returns 403 | Medium | Open | `GET /api/v1/tunnel/status/{id}` returns 403 for closed sessions (should return `{status: closed}`). Root cause: `verify_session_ownership()` applies `WHERE status='active'` before ownership check. Fix in `server/src/db/tunnel.rs:94-103`. |
+| S9 | | | | |
+
+## Tunnel Channels (Phase 2)
+
+On-demand capabilities layered on top of the tunnel session framework. Each channel is a typed WebSocket payload pair (request/response) routed by `channel_id` under an open `tech_session`. All channel operations are audited per Logging & Audit section.
+
+| # | Feature | Priority | Status | Notes |
+|---|---------|----------|--------|-------|
+| T1 | Terminal channel (interactive shell) | High | Open | `TunnelDataPayload::Terminal { command }` → `TerminalOutput { stdout, stderr, exit_code }` (types exist in `server/src/ws/mod.rs:310-319`, agent stub at `agent/src/transport/websocket.rs:408-434`). Implement via `tokio::process::Command` with configurable timeout (default 30s). 80% of field use cases. Ship before other channels. |
+| T2 | File channel (upload/download/rename/delete + tree browse) | High | Open | Covers D7. Stream file bytes in chunks over WS with progress. Path safety (no `..` traversal). Needs allowlist vs freeform decision. |
+| T3 | Registry channel (Windows) | Medium | Open | Covers D8. Read/write/create/delete keys + values. Use `winreg` crate. Gate to tenant admins only. |
+| T4 | Service channel (Windows services) | High | Open | Covers D9. List/start/stop/restart/change-startup-type. `windows-service` crate. |
+| T5 | Tech-side tunnel subscriber | High | Open | **Blocks all channels.** Browser currently has no mechanism to receive tunnel data from server. Design: `GET /api/v1/tunnel/stream/{session_id}` WebSocket + in-memory `HashMap<session_id, mpsc::Sender<TunnelData>>` pub-sub. |
+| T6 | Server-side forward path | High | Open | `server/src/ws/mod.rs:808-825` currently logs+drops incoming `AgentMessage::TunnelData`. Wire to T5 pub-sub + tunnel_audit INSERT. |
+| T7 | Working directory / shell choice / elevation decisions | High | Open | Terminal channel design decisions: cwd allowlist vs free-form; PowerShell vs cmd on Windows; admin elevation gating by role. |
+| T8 | Channel concurrency + rate limits | Medium | Open | Multiple channels in one session. Per-channel rate/quota. Output size cap (default 1 MB/command). |
+| T9 | | | | |
+
+## Logging, Audit & Observability
+
+Three-tier design decided 2026-04-15. Each tier has distinct purpose, storage, retention, and consumer.
+
+**Design principles:**
+- **Agent self-logging** uses OS-native mechanisms (no custom transport). Troubleshoot with familiar tools.
+- **Client machine health** via OS event log pulls. Feeds dashboard and alerting.
+- **Tunnel audit** captured directly to RMM DB. Non-negotiable, never scrubbed, designed for legal/compliance retention.
+
+| # | Feature | Priority | Status | Notes |
+|---|---------|----------|--------|-------|
+| L1 | Agent self-logging via OS-native sinks | High | Open | Windows Event Log (custom `GuruRMM-Agent` provider registered at install), Linux systemd/journald (`tracing` → stdout when run as unit), macOS unified log (`os_log` crate). Verbosity per-tenant configurable. Default INFO. |
+| L2 | Client event log pull + summarize | High | Open | Agent polls OS event log on schedule; ships filtered events to server `client_events` table. Windows: `Get-WinEvent -Level 1,2 -MaxEvents N`. Linux: `journalctl -p err --output json`. macOS: `log show --predicate 'messageType == error' --style json`. |
+| L3 | L2 cadence — default 15-min delta poll + on tunnel open/close | High | Open | Default 900s. On tunnel open: force delta pull so tech has fresh context. On tunnel close: force delta pull to capture anything tech's actions triggered. Configurable per-tenant in dashboard. |
+| L4 | L2 levels — default Critical + Error + Warning | High | Open | Configurable per-tenant. Default: Critical(1), Error(2), Warning(3). Separate "noisy" bucket (Info/Debug/Audit/Notification) pulled every 4h default. |
+| L5 | Tunnel audit — every tech action persisted | High | Open | Reuse existing `tunnel_audit` table (migration 010, unused today). Every command, file op, registry op, service op gets INSERT with session_id, channel_id, operation, details JSONB. No scrubbing — must retain sensitive input if a tech types it. |
+| L6 | Retention config | High | Open | `client_events`: 90 days default, tenant configurable. `tunnel_audit` (live): 90 days default, tenant configurable. `tunnel_audit` (archive): indefinite, system-level rotation to object storage. Agent self-logs follow OS-native retention policy. |
+| L7 | Tunnel audit archive rotation | High | Open | Monthly job: aged partitions of `tunnel_audit` → compressed JSONL or Parquet in S3/R2/MinIO. Naming: `tunnel_audit/tenant_id={uuid}/year={YYYY}/month={MM}.jsonl.gz`. Dashboard "deep search" endpoint queries archive on demand (Athena/DuckDB). |
+| L8 | Agent config push | High | Open | On agent WS connect, server sends `ServerMessage::Config { tenant_settings }`. Real-time updates when tenant admin changes settings in dashboard. Agent adjusts poll cadence + event level filters live without restart. |
+| L9 | Dashboard surfaces for L2 (client_events) | Medium | Open | Red-number badge on agent tile (count of unresolved errors last 24h). Time-sorted feed on agent detail page with filter/search. Acknowledge/dismiss individual events. |
+| L10 | Sensitive-data-at-rest protection | High | Open | `tunnel_audit` may contain unscrubbed credentials. Postgres TDE or full-disk encryption on server. Access to audit tables strictly admin-role-gated. Meta-audit: log every `SELECT` on `tunnel_audit` to separate table. Document in tech SOP: "every tunnel keystroke is logged." |
+| L11 | | | | |
+
+## Multi-tenancy / MSP SaaS
+
+Goal stated 2026-04-15: make this a marketable product for other MSPs. Multi-tenancy must be baked in from here on — adding `tenant_id` later would be a brutal migration.
+
+| # | Feature | Priority | Status | Notes |
+|---|---------|----------|--------|-------|
+| M1 | Core tenancy schema | High | Open | New tables: `tenants` (id, name, plan, status, created_at), `tenant_settings` (tenant_id, key, value JSONB), `msp_users` (superadmins across tenants), `tenant_users` (tech ↔ tenant join with role). Add `tenant_id UUID` FK to: `agents`, `tech_sessions`, `tunnel_audit`, `client_events`, `commands`, any other per-customer table. |
+| M2 | Tenant-scoped authorization | High | Open | JWT carries `tenant_id` + `role`. Every query must filter by tenant_id (middleware). Super-admin role bypasses for GuruRMM staff. Penalty for bugs here: data leakage across tenants. |
+| M3 | Tenant admin dashboard | High | Open | UI for MSP admins to configure their tenant settings (L3/L4/L6 cadences, levels, retention). Super-admin can override across tenants. |
+| M4 | Billing / licensing meter | Medium | Open | Per-agent-per-month is standard for RMM. Needs usage counter from day one. Consider Stripe Billing or manual invoicing to start. |
+| M5 | Data residency options | Low | Open | Some MSPs require on-prem or regional hosting. Architectural impact: deployment model (single-tenant vs multi-tenant DB), encryption key management. Not required for MVP. |
+| M6 | Tenant export API | Medium | Open | MSPs with SOC2/PCI customers will need to export their tenant's audit trail. `GET /api/v1/tenants/{id}/export` producing JSONL or Parquet. Self-service for portability. |
+| M7 | Onboarding flow | High | Open | MSP signs up → tenant provisioned → first site created → install link generated → agent installs → first heartbeat → onboarding complete. End-to-end wizard. |
+| M8 | | | | |
 
 ## Infrastructure / Operations
 
@@ -57,6 +139,108 @@ Tracked list of desired features, improvements, and changes. Used to evaluate wh
 
 ---
 
+## Modular Architecture & Public APIs
+
+Goal stated 2026-04-15: the product should be modular from inception. Future modules under consideration: PSA/CRM, remote syslog aggregation, backups, likely more. Both first-party (us) and eventually third-party (other developers, customers) should be able to build modules against stable, versioned interfaces. End users should also have API access to automate against their own data.
+
+**Architectural principles:**
+- **Core is thin + opinionated.** Tenants, agents, auth, audit, command dispatch, tunnel framework — that's the "kernel." Everything else is a module.
+- **Modules own their data.** Each module owns a schema namespace (`psa_*`, `backup_*`, `syslog_*`) and never writes directly to another module's tables. Cross-module data access goes through module-exposed APIs.
+- **Event bus for cross-cutting communication.** Agent.online, tunnel.opened, command.completed, client_event.received — core publishes, any module subscribes.
+- **Public API is a first-class product surface**, not an afterthought. OpenAPI spec, semver-versioned, rate-limited, key-authenticated, documented.
+- **Boundary discipline:** if it's tempting to reach across a module boundary, that's a signal to add an API there instead. Breaking this discipline once kills the modularity.
+
+| # | Feature | Priority | Status | Notes |
+|---|---------|----------|--------|-------|
+| X1 | Core vs. module boundary definition | High | Open | Document what's "core" (tenants, agents, auth, audit, command dispatch, tunnel framework, bootstrap) vs. what's a module (everything else). Codify via separate crates / modules in the Rust workspace (`core/`, `modules/psa/`, `modules/backups/`, etc.). Enforce via build system — module code cannot `use` private core internals, only the exposed `core::api::*` surface. |
+| X2 | Module manifest / registration | High | Open | Each module ships a `module.toml` declaring: name, version, provides (APIs exposed), consumes (events/APIs used), permissions required (read_agents, write_commands, read_audit, etc.). Loaded at server startup; dashboard reflects installed modules. |
+| X3 | Event bus | High | Open | NATS JetStream or Redis Streams. Every significant core action emits a typed event (`agent.online`, `agent.offline`, `tunnel.opened`, `tunnel.closed`, `command.completed`, `client_event.received`, `tenant.created`). Modules subscribe via the bus, not via direct core calls. Decouples timing + enables async modules. |
+| X4 | Module-to-core APIs | High | Open | Core exposes a stable in-process API for modules: `core::agents::list(tenant_id)`, `core::commands::enqueue(...)`, `core::audit::record(...)`. Versioned like `core_api_v1`, `core_api_v2`. Modules declare which version they require. |
+| X5 | Module-to-module APIs | Medium | Open | Modules can expose their own APIs for other modules to consume. Example: PSA module exposes `psa::tickets::create()` which a Backups module could call when a backup fails. All via the module registry — no direct imports. |
+| X6 | Public REST API (for end users + integrations) | High | Open | Versioned under `/api/public/v1/`. OpenAPI 3.1 spec auto-generated. Rate-limited per API key. Scoped API keys (read-only / write / admin). Separate from internal `/api/v1/` used by dashboard. Publish spec at `/api/public/v1/openapi.json`. |
+| X7 | API key management | High | Open | Dashboard UI: tenants create/revoke/rotate API keys, scope per key, view last-used and usage stats. Keys carry tenant_id. JWT session tokens (for dashboard) are separate from API keys (for machines). |
+| X8 | Public webhook subscriptions | High | Open | Tenants subscribe to events via webhook URL. Event bus (X3) feeds a delivery worker that signs payloads (HMAC), retries with backoff, tracks delivery status in DB. Lets customers integrate without polling. |
+| X9 | Third-party module sandbox | Medium | Open | Future work. Options: (a) WebAssembly modules loaded at runtime with capability-based access to core APIs; (b) signed OCI container images run as sidecars with mTLS. (a) is better UX but maturity risk. (b) is ops-heavy but proven. Decide when third-party demand is real. |
+| X10 | Module billing isolation | Medium | Open | Each module can have independent pricing (PSA seat-based, Backups GB-based, RMM per-agent). Core billing meter (M4) becomes per-module, aggregates to tenant invoice. Enable tenants to subscribe to some modules but not others. |
+| X11 | Module upgrade independence | Medium | Open | Modules version independently of core. Core API versioning (X4) lets modules pin `core_api_v2` and survive core updates. Dashboard shows which modules need upgrades for a new core release. |
+| X12 | Module discoverability / marketplace | Low | Open | Eventually: marketplace UI for MSPs to browse/install first- and third-party modules. Signed+reviewed entries only. Revenue share for third-party developers. Many moons away, design constraint for now: don't paint ourselves into a corner. |
+| X13 | | | | |
+
+### Module candidates currently in mind
+
+Capture these now so the core API design has concrete use cases to validate against:
+
+- **PSA/CRM module** — tickets, time tracking, contracts, invoicing. Likely largest module, heaviest DB load. Consumes: `agent.online`, `client_event.received`, `command.completed`. Exposes: `psa::tickets::create|assign|close`, `psa::time::log`.
+- **Remote Syslog module** — aggregates syslog/Windows Event Log from customer devices to a central searchable store. Consumes: `client_event.received`. Exposes: `syslog::query|subscribe`. Heavy ingest.
+- **Backups module** — schedules, monitors, reports on backup jobs (Veeam, Datto, Acronis, Synology, etc.). Consumes: integrations with third-party backup products (pull). Exposes: `backups::status|history|alert`. Compliance-sensitive.
+- **Patch management** — track OS + app patch levels, schedule installs, report compliance.
+- **Documentation (IT Glue-style)** — customer environment docs, credential vault, runbooks. Deep integration with PSA (customer entity shared).
+- **Remote access** — already covered by core tunnel framework; could grow into its own "pro" module with session recording, MFA-gated elevation, etc.
+- **Network monitoring** — SNMP/ping monitoring of non-agent devices (switches, printers, UPSs).
+
+## Protocol Versioning & Stale-Agent Recovery
+
+Problem surfaced 2026-04-15: as the codebase evolves (multi-tenancy pivot, tunnel channels, new message types), long-offline agents will return to find the wire format they knew is gone. Without an upgrade lane, those agents become zombies — visible in the dashboard as "offline for 47 days," never self-heal, require manual intervention (RDP in, uninstall, reinstall).
+
+Concrete example: Scileppi VP laptop offline for days. When it wakes up and tries to check in with v0.6.0 against a server that by then expects v0.9.x protocol, we need the server to say "I see you, you're old, here's how to update yourself" — and have the agent auto-comply.
+
+**Design principle:** the bootstrap/hello path is sacred. It must never break, even across major protocol revisions. All other endpoints and message shapes are allowed to change. An agent that can still reach `/hello` can always recover.
+
+| # | Feature | Priority | Status | Notes |
+|---|---------|----------|--------|-------|
+| V1 | Protocol version negotiation on connect | High | Open | Agent sends `{agent_version, protocol_version, os, arch}` as first message. Server responds with `{server_version, min_supported_protocol, latest_protocol, action}` where action ∈ {`proceed`, `upgrade_required`, `rejected`}. WebSocket subprotocol header is one delivery option; a dedicated HTTP hello endpoint is another. Pick one, then never change its shape. |
+| V2 | Stable bootstrap endpoint | High | Open | `POST /api/v1/bootstrap/hello` that accepts the agent handshake forever. Contract: input schema is additive-only (new optional fields OK, never rename/remove), output shape is additive-only. Agents as old as v0.1 must be able to hit this and get meaningful response. |
+| V3 | Compat shim layer per old protocol version | High | Open | When an old agent checks in, server translates between the old wire format and current internal types. Shim lives in `server/src/compat/v{N}.rs`. Each shim documents: which protocol versions it supports, what adapters it provides, planned removal date. |
+| V4 | Server-initiated forced upgrade instruction | High | Open | When handshake returns `action: upgrade_required`, response also includes `update_url`, `update_checksum`, `update_args`, and optional `restart_policy`. Agent treats this as highest-priority command, bypasses normal command queue, upgrades + relaunches itself. |
+| V5 | Agent self-update atomic rename (verify) | High | Exists (hardening needed) | Already done per 2026-04-01 ADR. Audit against V4 flow: does current updater handle "tell me exactly which version to install" vs. "upgrade to latest"? May need parameterization. |
+| V6 | Per-version support matrix + sunset policy | High | Open | Dashboard surface: table showing N agents per protocol version per tenant. Automated sunset: when a protocol version has 0 live agents for 60 days across all tenants, flag compat shim for removal in next release. Manual override to force-remove earlier. |
+| V7 | Agent version pinning per tenant | Medium | Open | MSP can opt tenants into "stable" (N-1), "current" (latest), or "beta" (preview) update channels. Controls auto-update rollout pace across their fleet. |
+| V8 | Late check-in handling: accept then command | High | Open | On stale-agent connect: (a) accept the handshake via compat shim, (b) record the connect event in audit, (c) immediately enqueue the upgrade command, (d) agent executes before any other work. Dashboard shows agent as "upgrading" briefly before "online". |
+| V9 | Graceful protocol deprecation warnings | Medium | Open | When an agent connects on a deprecated (but still supported) protocol, server sends a warning field in every response. Agent logs it. Gives MSPs lead time to upgrade their fleet before hard-removal. |
+| V10 | Rollback path for bad upgrades | High | Open | If v0.N upgrade bricks agents, bootstrap endpoint must let an operator mark v0.N `action: downgrade_required` and ship an older binary. Requires keeping old binaries in `/var/www/gururmm/downloads/` with pinned checksums. |
+| V11 | | | | |
+
+## Certificates & Trust
+
+Code signing and TLS/trust certificates required to ship + operate the product without install-time friction. Decisions 2026-04-15.
+
+| # | Item | Priority | Status | Cost | Notes |
+|---|------|----------|--------|------|-------|
+| C1 | Azure Trusted Signing — Windows agent + installer | High | In progress (2026-04-15) | ~$9.99/mo + per-sig fee | Hosted signing service. Bypasses hardware-token requirement that took effect June 2023. Public Trust level requires 3+ yrs business existence; Private Trust available immediately but limited usefulness. Identity verification via Microsoft takes days. See setup steps in session-logs/2026-04-15. |
+| C2 | Apple Developer Program — macOS agent notarization | High | Open | $99/yr | Developer ID Application + Installer certs; notarization via `xcrun notarytool`; Hardened Runtime entitlements; ticket stapling for offline installs. Enrollment can take days — start early. |
+| C3 | GPG signing — Linux .deb / .rpm packages | High | Open | Free | Generate key pair, publish pubkey at a stable URL, sign packages with `debsign`/`rpmsign`, host signed apt/yum repo with proper `Release`/`repomd.xml`. |
+| C4 | Timestamping — all signed artifacts | High | Open | Free | Use DigiCert or Sectigo public timestamp servers so signatures remain valid after cert rotation. Verify in CI that every signed binary has a valid timestamp. |
+| C5 | TLS automation for own domains | High | Done | Free | Cloudflare + Let's Encrypt already in place for `rmm-api.azcomputerguru.com`. Wildcard for `*.gururmm.com` when that domain lights up. |
+| C6 | Per-Partner white-label custom domains | Medium | Open | ~$7/mo/domain via CF-for-SaaS, or DIY with ACME DNS-01 | Partners want `rmm.theirbrand.com`. Decide: host certs ourselves via ACME DNS-01 + Cloudflare API, or use Cloudflare for SaaS. Defer until first Partner asks. |
+| C7 | Agent-to-server mTLS (enterprise option) | Low | Open | Internal CA + time | Self-signed CA + per-agent client certs. Bootstrap enrolls agent and issues cert scoped to `agent_id`. Adds install complexity. Defer until an enterprise customer demands it. |
+| C8 | SBOM + Sigstore/cosign provenance | Medium | Open | Free | Auto-generate CycloneDX or SPDX SBOM per release. `cosign` sign artifacts + container images. Important for SOC2-conscious MSPs evaluating supply chain. |
+| C9 | Windows Defender / vendor FP submission runbook | Medium | Open | — | Despite valid signing, heuristic engines flag new binaries. Keep a runbook with submission portal links (Microsoft Security Intelligence, Malwarebytes, etc.). |
+| C10 | Email sending trust: DKIM / SPF / DMARC | Medium | Open | Free | Required when PSA module sends ticket notifications. Set up on sending domain; per-Partner if white-labeled email is a feature. |
+| C11 | WHQL driver signing | Deferred | Open | $$$ + weeks turnaround | Only if we ship a kernel driver. Avoid this path — use user-mode alternatives first. |
+| C12 | | | | | |
+
+## Decisions Log
+
+Short record of why things are the shape they are. Append, don't edit.
+
+**2026-04-15 — Tunnel Phase 1 verified live.** End-to-end test from off-LAN workstation via `rmm-api.azcomputerguru.com`. Open/status/close lifecycle works. Confirmed nginx proxies `/api/*` (not just `/downloads/`). See session-logs/2026-04-15-session.md.
+
+**2026-04-15 — Logging split into three tiers.** Decided against a single custom log transport. Agent self-logging to OS-native sinks (Event Viewer / journald / os_log). Client machine health via OS event log pulls. Tunnel audit direct to RMM DB. Rationale: sysadmins can troubleshoot with familiar tools; only high-value audit data hits our DB.
+
+**2026-04-15 — Tunnel audit is never scrubbed.** If a tech types a password during a session, it gets stored. Purpose is to audit tech behavior, and scrubbing would undermine that. Offsetting controls: encryption at rest, admin-role-gated access, meta-audit of log views, tech SOP documentation. See L10.
+
+**2026-04-15 — Multi-tenancy from day one.** Target market is MSPs reselling this product. Adding `tenant_id` retroactively after feature growth is a brutal migration; baking it in now is cheap. Every new table gets `tenant_id` FK from here forward.
+
+**2026-04-15 — Poll cadences.** 15-min delta + on-tunnel-open/close for critical+error+warning. 4h bulk for info/debug/audit/notification. All tenant-configurable.
+
+**2026-04-15 — Retention.** 90 days default for tenant-visible tables. Indefinite system-level for `tunnel_audit` with object-storage archive after the tenant-visible window. Legal/compliance contexts (HIPAA 6yr, PCI 1yr) handled by per-tenant extended retention configs.
+
+**2026-04-15 — Hierarchy terminology locked.** Platform > Partner (MSP, DB: tenant_id) > Client > Site > Agent. API and UI say "Partner"; DB says `tenant_id`. No "sub-tenant", no ambiguous "customer". Department/OU tier deferred. MSPs can white-label labels via JSONB overrides. See Terminology section at top of this file.
+
+**2026-04-15 — Modular architecture from day one.** Core = tenants + agents + auth + audit + commands + tunnel framework + bootstrap. Everything else = module. Modules own their schema namespace, never touch each other's tables, communicate via event bus (X3) and versioned module APIs (X4/X5). Public REST API (X6) separate from internal dashboard API. Webhook subscriptions (X8) for customer integrations. Third-party modules via WASM or signed containers — deferred but design-constrained now. Concrete module candidates: PSA/CRM, remote syslog, backups, patch management, IT-Glue-style docs, network monitoring. See X1-X12.
+
+**2026-04-15 — Bootstrap endpoint is sacred.** Protocol version negotiation via a single `/api/v1/bootstrap/hello` endpoint whose input/output are additive-only forever. Every other endpoint/message is free to evolve. Enables late-arriving agents (Scileppi VP example: offline for days, wakes up to find a newer server protocol) to reconnect, get accepted, and receive an automatic upgrade instruction. Compat shim layer per old protocol version with automated sunset policy when fleet-wide usage hits zero. See V1-V10.
+
 ## Rewrite Assessment
 
 **Criteria for rewrite:**
@@ -64,4 +248,4 @@ Tracked list of desired features, improvements, and changes. Used to evaluate wh
 - If the tech stack is fundamentally wrong for the goals
 - If accumulated tech debt makes changes unreasonably slow
 
-**Current assessment:** TBD -- add features above first, then evaluate.
+**Current assessment (2026-04-15):** The multi-tenancy pivot means a schema refactor is unavoidable (add `tenant_id` everywhere, tenancy-aware auth middleware). This is additive, not a rewrite. Rust + Axum + Postgres + WebSocket stack remains fit for purpose. Current code is a solid foundation. No rewrite planned; structural additions tracked above.

projects/msp-tools/guru-rmm/session-logs/2026-04-15-session.md

@@ -0,0 +1,162 @@
+# GuruRMM Session Log — 2026-04-15
+
+## Context
+End-to-end test of the Tunnel Phase 1 lifecycle, triggered opportunistically
+while troubleshooting SSH flakiness on AD2 (Dataforth project). No code
+changes — exercised the production API from an off-LAN workstation via the
+public Cloudflare endpoint (`rmm-api.azcomputerguru.com`).
+
+## What worked
+
+| Step | Endpoint | Result |
+|---|---|---|
+| Login | `POST /api/auth/login` | 200, token returned |
+| List agents | `GET /api/agents` | 6 agents, AD2 and DESKTOP-0O8A1RL online on v0.6.0 |
+| Open tunnel | `POST /api/v1/tunnel/open` (agent_id=AD2 `d28a1c90-47d7-448f-a287-197bc8892234`) | 200, `{session_id: 0682a80c-a899-403b-9473-aaaed50e4aba, status: active}` |
+| Status while active | `GET /api/v1/tunnel/status/{id}` | 200, full session record (opened_at, last_activity, agent_id) |
+| Close tunnel | `POST /api/v1/tunnel/close` | 200, `{status: closed}` |
+
+## Findings (actionable)
+
+### 1. Status endpoint returns 403 after close
+`GET /api/v1/tunnel/status/{id}` against a just-closed session returns
+`403 Forbidden — "Session not found or not owned by user"` instead of
+`{status: closed}`. Root cause likely that the `WHERE status = 'active'`
+filter (from `idx_tech_sessions_active` — see CONTEXT.md line 256) is applied
+to the status lookup in addition to the ownership check, so closed sessions
+fail ownership verification and fall through to the 403 branch.
+
+**Fix:** separate the existence lookup from the ownership check. If the
+session exists but belongs to the requesting tech, return the closed record
+rather than masking it as a permission error.
+
+Location to inspect: `server/src/api/tunnel.rs` (status handler) and/or
+`server/src/db/tunnel.rs` (session fetch query).
+
+### 2. Agent writes no logs
+`gururmm-agent.exe 0.6.0` on AD2 produces no files in
+`C:\Program Files\GuruRMM\`, `C:\ProgramData\GuruRMM\`, nor any Windows
+Application Event Log entries under provider `gururmm*`. This made it
+impossible to confirm the agent-side state transition
+(`Heartbeat → Tunnel`) or receipt of `TunnelReady` during the test.
+
+**Fix:** add a log target in `agent/src/main.rs` (env_logger or tracing
+with a rolling file appender) writing to
+`C:\ProgramData\GuruRMM\agent.log`. Optionally also emit critical events
+(tunnel open/close, update success/failure) to the Windows Event Log via
+`eventlog` crate.
+
+### 3. Phase 2 gap confirmed against a real use case
+Live need: run a couple of diagnostic commands on AD2 (sshd flapping
+sporadically on port 22, no process crash in Event Log; want to investigate
+firewall/Defender events from the server side). With no channels, the
+tunnel's only utility today is proving the session layer works. The actual
+remote-operate capability still depends on Phase 2.
+
+**Priority order for Phase 2 channels** (based on what would have been useful
+here):
+1. **Terminal channel** first — unlocks 80% of field use cases (log tails,
+   `Get-Service`, `Restart-Service`, `Get-WinEvent`).
+2. **Service channel** second — tight scope, high value for "restart sshd".
+3. **File channel** third — needed but rarely urgent; SFTP already exists.
+4. **Registry channel** last — niche, can defer.
+
+## What Else We Observed
+
+- The public tunnel chain `rmm-api.azcomputerguru.com` → Cloudflare → nginx
+  → API (3001) proxies `/api/*` correctly. The docs in CONTEXT.md implied
+  nginx only served `/downloads/`; confirmed today that it also proxies API
+  paths, which is why off-LAN admin usage works.
+- AD2 agent start time `2026-04-11 22:09` corresponds to last reboot of
+  AD2; the agent has not restarted since despite sshd port flaps (sshd PID
+  4012 also continuously running since same moment). Confirms the tunnel
+  infrastructure and the RMM agent are stable; the sshd flap is a separate
+  network-layer issue unrelated to GuruRMM.
+
+## Credentials Used
+
+- **Admin Email:** admin@azcomputerguru.com
+- **Admin Password:** GuruRMM2025
+- **Public API:** https://rmm-api.azcomputerguru.com
+
+**Note:** `op read "op://Infrastructure/GuruRMM Server/Admin Password"`
+returned a stale value (`ClaudeAPI2026!@#`) that fails login. The
+2026-04-14 session log documents the current password as `GuruRMM2025`.
+1Password entry should be updated to match.
+
+## Next Steps
+
+1. Update 1Password `Infrastructure/GuruRMM Server` entry — set
+   `Admin Password` field to `GuruRMM2025` to match what server accepts.
+2. Fix `/api/v1/tunnel/status/{id}` for closed sessions (see Finding 1).
+3. Add file/event-log output to agent (see Finding 2).
+4. Begin Phase 2 — Terminal channel first.
+
+---
+
+## Update (evening session): Roadmap evolution + Azure Trusted Signing setup
+
+Substantial architectural planning session. Product direction shifted from "single-tenant RMM tool" to "multi-tenant SaaS for MSPs." Roadmap updated significantly to reflect.
+
+### Roadmap additions to ROADMAP.md
+
+1. **Terminology (canonical)** — locked in the 5-tier hierarchy: Platform → Partner (DB: tenant_id) → Client → Site → Agent. API/UI says "Partner"; DB column is `tenant_id`. API path convention `/api/public/v1/partners/{pid}/clients/{cid}/sites/{sid}/agents/{aid}`. Event topics like `agent.online`, `partner.upgraded`. Full table + rules at top of ROADMAP.md.
+
+2. **Tunnel Channels (Phase 2)** — T1-T8 tracking Terminal/File/Registry/Service channels + tech-side subscriber (T5 is gating dep — browser currently has no way to receive tunnel data, `server/src/ws/mod.rs:808-825` discards incoming `AgentMessage::TunnelData`).
+
+3. **Logging, Audit & Observability** — L1-L10 three-tier design:
+   - Agent self-logging via OS-native sinks (Windows Event Log custom provider, Linux journald, macOS os_log)
+   - Client machine health via OS event log pulls — default 15-min delta + force-pull on tunnel open/close; default levels Critical+Error+Warning for delta, 4h bulk for Info/Debug/Audit/Notification; all tenant-configurable
+   - Tunnel audit direct to DB table `tunnel_audit` (already exists, unused) — no scrubbing, sensitive input captured intentionally for tech-behavior audit; 90-day tenant-visible retention default; indefinite system archive to object storage
+   - Agent config push via `ServerMessage::Config` on connect + real-time when tenant admin changes settings
+
+4. **Multi-tenancy / MSP SaaS (M1-M7)** — tenant_id on every table from now forward, tenancy-aware auth middleware, tenant admin dashboard, per-agent/month billing meter, data residency options, tenant export API, onboarding wizard.
+
+5. **Modular Architecture & Public APIs (X1-X12)** — core vs. module boundary, event bus (NATS JetStream or Redis Streams), module manifest, module-to-core + module-to-module versioned APIs, public REST API `/api/public/v1/` with OpenAPI spec + scoped API keys, webhook subscriptions, WASM or OCI sandbox for third-party modules (deferred), per-module billing. Concrete module candidates documented: PSA/CRM, Remote Syslog, Backups, Patch Mgmt, IT-Glue-style Docs, Network Monitoring.
+
+6. **Protocol Versioning & Stale-Agent Recovery (V1-V10)** — `/api/v1/bootstrap/hello` declared **sacred** (additive-only forever). Compat shim layer per old protocol version at `server/src/compat/v{N}.rs`. Server-initiated forced-upgrade instruction. Per-tenant update channels (stable/current/beta). Auto-sunset policy when old version fleet hits zero. Rollback path via `action: downgrade_required`. Concrete motivating example: Scileppi VP laptop offline for days — must be able to reconnect, get accepted, auto-upgrade.
+
+7. **Certificates & Trust (C1-C11)** — full cost + priority matrix. C1: Azure Trusted Signing for Windows (Public Trust). C2: Apple Developer Program. C3: GPG for Linux. C4-C11: TLS automation, mTLS, SBOM, FP submissions, DKIM.
+
+8. **Decisions Log** — appended rationale entries for every 2026-04-15 decision so future sessions don't re-litigate.
+
+### CONTEXT.md anti-patterns added
+
+- "DO NOT make breaking changes to `/api/v1/bootstrap/hello`" — additive-only forever
+- "DO NOT cross module boundaries by importing another module's internals" — event bus or exposed APIs only
+- Hierarchy terminology table added to anti-patterns block (canonical reference)
+
+### Azure Trusted Signing — provisioned and IV submitted
+
+**Business identity confirmed** via D&B profile lookup: `Arizona Computer Guru LLC` (D-U-N-S `00-566-1506` / `005661506`), 7437 E 22ND St, Tucson AZ 85710, (520) 304-8300, mike@azcomputerguru.com. 25+ years operating history → Public Trust eligible (>3yr threshold).
+
+**Provisioned in subscription `Basic` (`e507e953-2ce9-4887-ba96-9b654f7d3267`):**
+- Resource group: `gururmm-signing-rg` (westus2)
+- Trusted Signing Account: `gururmm-signing`
+- Account URI: `https://wus2.codesigning.azure.net/`
+- SKU: Basic (~$9.99/mo billing started 2026-04-16 00:16 UTC)
+
+**RBAC granted:**
+- `mike@azcomputerguru.com` → role `Artifact Signing Identity Verifier` at account scope
+
+**Identity Validation submitted:**
+- IV ID: `03028768-f611-4904-aa58-c755020f436a`
+- Status: `In Progress` (Microsoft review, 1-5 business days typical)
+- Submitted name: `Arizona Computer Guru LLC` (state filing); D&B record has older `COMPUTER GURU` Corporation — may need to update D&B profile for consistency
+- Primary email: mike@; Secondary: admin@azcomputerguru.com
+- Microsoft may call 520-304-8300 — voicemail should identify Computer Guru
+
+**Pending (blocks on IV approval):**
+- Certificate Profile creation: `az trustedsigning certificate-profile create --resource-group gururmm-signing-rg --account-name gururmm-signing --profile-name gururmm-public-trust --profile-type PublicTrust --identity-validation-id 03028768-f611-4904-aa58-c755020f436a`
+- Signing role assignment: `Trusted Signing Certificate Profile Signer` to CI build principal
+- Local tooling install: Windows SDK (for signtool.exe), Microsoft.Trusted.Signing.Client NuGet package
+
+**All details persisted to vault:** `D:\vault\services\azure-trusted-signing.sops.yaml` (encrypted).
+
+### Action items for next session
+
+1. Check IV status — portal → Trusted Signing Accounts → gururmm-signing → Identity Validation
+2. If approved → run the cert profile create command (already staged in vault)
+3. If Microsoft flags legal name mismatch: reply with AZ Corp Commission LLC Articles; update D&B record
+4. Start signtool.exe + dlib integration in a local scratch project
+5. Meanwhile, fix the two backlog items (tunnel status 403 bug, agent logging) — they're both independent of the Azure work and small PRs

projects/newsletter/flash-sale-newsletter.html

@@ -0,0 +1,296 @@
+<!--[if mso]>
+<style>table, td {font-family: Georgia, serif !important;}</style>
+<![endif]-->
+
+<!-- OUTER WRAPPER -->
+<table width="100%" cellpadding="0" cellspacing="0" border="0" style="background-color: #f4f1ec;">
+<tr>
+<td align="center" style="padding: 20px 10px;">
+
+<!-- INNER CONTAINER -->
+<table width="640" cellpadding="0" cellspacing="0" border="0" style="background-color: #ffffff; font-family: Georgia, 'Times New Roman', serif; color: #2a2a2a; line-height: 1.6;">
+
+<!-- HEADER BAR -->
+<tr>
+<td style="background-color: #1a1a1a; padding: 22px 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td>
+<span style="font-size: 24px; font-weight: bold; color: #ffffff; letter-spacing: 0.5px;">Arizona Computer Guru</span>
+</td>
+<td align="right" style="vertical-align: bottom;">
+<span style="font-size: 12px; color: #999999; letter-spacing: 2px; text-transform: uppercase;">Est. 2001 &bull; Tucson, AZ</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- RED ACCENT LINE -->
+<tr>
+<td style="background-color: #c0392b; height: 4px; font-size: 4px; line-height: 4px;">&nbsp;</td>
+</tr>
+
+<!-- FLASH SALE BANNER -->
+<tr>
+<td style="background-color: #1a1a1a; padding: 30px 35px; text-align: center;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td align="center">
+<span style="font-size: 11px; letter-spacing: 4px; color: #c0392b; text-transform: uppercase; font-weight: bold;">Limited Availability &bull; One Day Only</span>
+</td>
+</tr>
+<tr>
+<td align="center" style="padding-top: 12px;">
+<span style="font-size: 34px; font-weight: bold; color: #ffffff;">FLASH SALE</span>
+</td>
+</tr>
+<tr>
+<td align="center" style="padding-top: 6px;">
+<span style="font-size: 18px; color: #cccccc;">Prepaid Labor Blocks</span>
+</td>
+</tr>
+<tr>
+<td align="center" style="padding-top: 18px;">
+<table cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="background-color: #c0392b; padding: 14px 30px; text-align: center;">
+<span style="font-size: 28px; font-weight: bold; color: #ffffff;">$100/hr</span><br />
+<span style="font-size: 13px; color: #f5c6cb; text-decoration: line-through;">normally $150/hr</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+<tr>
+<td align="center" style="padding-top: 14px;">
+<span style="font-size: 14px; color: #888888;">10-Hour Blocks &bull; Save $500 Per Block</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 30px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- FLASH SALE BODY -->
+<tr>
+<td style="padding: 0 35px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 17px; margin: 0 0 16px 0; color: #2a2a2a;">Hi there,</p>
+
+<p style="font-size: 17px; margin: 0 0 16px 0; color: #2a2a2a;">We don't do these often, so here's the short version.</p>
+
+<p style="font-size: 17px; margin: 0 0 20px 0; color: #2a2a2a;">For <strong>one day only</strong>, we're offering 10-hour prepaid labor blocks at <strong>$100/hour</strong>. That's <strong>$1,000 per block</strong> instead of the usual $1,500. If you've been putting off that server migration, network upgrade, or office buildout -- this is the time to lock in the hours.</p>
+</td>
+</tr>
+
+<!-- DETAILS BOX -->
+<tr>
+<td style="padding: 0 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="border-left: 4px solid #c0392b; background-color: #faf9f7; padding: 20px 25px; font-family: Georgia, 'Times New Roman', serif;">
+<span style="font-size: 15px; font-weight: bold; color: #1a1a1a;">The Details:</span><br />
+<span style="font-size: 15px; color: #2a2a2a; line-height: 1.8;">
+&#8226;&nbsp; 10-hour blocks at $100/hr (normally $150/hr)<br />
+&#8226;&nbsp; Limit: 4 blocks per client (40 hours max)<br />
+&#8226;&nbsp; Hours <strong>never expire</strong><br />
+&#8226;&nbsp; Use them for anything -- projects, support, on-site work<br />
+&#8226;&nbsp; <strong>One day only.</strong> When it's over, it's over.
+</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 20px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- CTA BUTTON -->
+<tr>
+<td align="center" style="padding: 0 35px 10px 35px;">
+<table cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="background-color: #c0392b; padding: 14px 40px; text-align: center;">
+<a href="tel:5203048300" style="font-family: Georgia, 'Times New Roman', serif; font-size: 18px; font-weight: bold; color: #ffffff; text-decoration: none;">Call 520.304.8300 to Claim Yours</a>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<tr>
+<td align="center" style="padding: 0 35px;">
+<p style="font-size: 14px; color: #888; margin: 6px 0 0 0; font-family: Georgia, 'Times New Roman', serif;">or reply to this email &bull; first come, first served</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 30px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- SECTION DIVIDER -->
+<tr>
+<td style="padding: 0 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="border-top: 2px solid #e8e4de; height: 1px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- RADIO SHOW -->
+<tr>
+<td style="padding: 0 35px; font-family: Georgia, 'Times New Roman', serif;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="background-color: #1a1a1a; padding: 4px 12px; display: inline-block;">
+<span style="font-size: 11px; letter-spacing: 3px; color: #c0392b; text-transform: uppercase; font-weight: bold;">Now Returning</span>
+</td>
+</tr>
+</table>
+<p style="font-size: 22px; font-weight: bold; margin: 12px 0 10px 0; color: #1a1a1a;">The Computer Guru Show</p>
+<p style="font-size: 16px; margin: 0 0 12px 0; color: #2a2a2a;">After a five-year break, we're bringing the radio show back. If you listened before, you know the format -- real talk about technology, security threats, and what actually matters for your business. No jargon, no fluff. New episodes are in production now for Season 11.</p>
+<p style="font-size: 16px; margin: 0 0 5px 0; color: #2a2a2a;">194 classic episodes are already archived at <a href="https://radio.azcomputerguru.com" style="color: #c0392b; font-weight: bold;">radio.azcomputerguru.com</a></p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- SECTION DIVIDER -->
+<tr>
+<td style="padding: 0 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="border-top: 2px solid #e8e4de; height: 1px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- SECURITY -->
+<tr>
+<td style="padding: 0 35px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 22px; font-weight: bold; margin: 0 0 12px 0; color: #1a1a1a;">Are You Actually Protected?</p>
+
+<p style="font-size: 16px; margin: 0 0 16px 0; color: #2a2a2a;">Two things worth knowing about:</p>
+
+<p style="font-size: 16px; margin: 0 0 16px 0; color: #2a2a2a;"><strong style="color: #1a1a1a;">Penetration Testing.</strong> We'll attack your network the same way a real threat actor would -- then hand you a detailed report of what we found and how to fix it. Most businesses have no idea what's actually exposed until someone shows them. We're offering a <strong>free security risk assessment</strong> that includes an external vulnerability scan, dark web search for your company's compromised credentials, and a simulated phishing test on your staff. No obligation, no sales pitch -- just a written report with a risk score. Call us or reply to schedule yours.</p>
+
+<p style="font-size: 16px; margin: 0 0 5px 0; color: #2a2a2a;"><strong style="color: #1a1a1a;">Advanced Antivirus for GPS Subscribers.</strong> If you're on our GPS-Pro or GPS-Advanced plans, you already have access to enterprise-grade EDR (Endpoint Detection &amp; Response) that goes well beyond what traditional antivirus catches. We're talking behavioral analysis, ransomware rollback, and real-time threat intelligence. If you're still on GPS-Basic and want to step up, now's a good time to talk about it.</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- SECTION DIVIDER -->
+<tr>
+<td style="padding: 0 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="border-top: 2px solid #e8e4de; height: 1px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- VOIP -->
+<tr>
+<td style="padding: 0 35px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 22px; font-weight: bold; margin: 0 0 12px 0; color: #1a1a1a;">Still Overpaying for Phone Service?</p>
+
+<p style="font-size: 16px; margin: 0 0 16px 0; color: #2a2a2a;">Our ACG-Voice (powered by PacketDial) business phone system starts at <strong>$22/user/month</strong> with unlimited calling, voicemail-to-email, and a softphone app so your team can take calls from anywhere. The Standard tier at $28/user adds desk phone support, call queues, and ring groups -- everything a real office needs without the enterprise price tag.</p>
+
+<p style="font-size: 16px; margin: 0 0 5px 0; color: #2a2a2a;">We handle the setup, port your existing numbers, and support the whole thing. If your current provider is nickel-and-diming you on features, let's have a conversation.</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- SECTION DIVIDER -->
+<tr>
+<td style="padding: 0 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="border-top: 2px solid #e8e4de; height: 1px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- WEB DEV -->
+<tr>
+<td style="padding: 0 35px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 22px; font-weight: bold; margin: 0 0 12px 0; color: #1a1a1a;">Need a Website That Doesn't Embarrass You?</p>
+
+<p style="font-size: 16px; margin: 0 0 5px 0; color: #2a2a2a;">We build and host business websites -- clean, fast, and actually maintained. Hosting starts at $15/month. If your site hasn't been touched since 2019 or you're paying a fortune for something that loads like it's on dial-up, we should talk. We also handle email hosting, SSL, and e-commerce if you need it.</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- SECTION DIVIDER -->
+<tr>
+<td style="padding: 0 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="border-top: 2px solid #e8e4de; height: 1px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 25px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- CLOSING -->
+<tr>
+<td style="padding: 0 35px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 16px; margin: 0 0 5px 0; color: #2a2a2a;">That's it. No filler. If any of this is relevant to you, pick up the phone or hit reply. We're local, we answer, and we don't waste your time.</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 25px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- SIGNATURE -->
+<tr>
+<td style="padding: 0 35px 30px 35px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="margin: 0 0 5px 0;"><span style="font-size: 20px;">Thanks,</span></p>
+<p style="margin: 0;"><span style="font-size: 20px;"> Michael Swanson<br />Owner<br /> </span><a href="http://www.azcomputerguru.com"><span style="font-size: 20px;">www.azcomputerguru.com</span></a> <span style="font-size: 20px;"><br /> phone:&nbsp;520.304.8300</span> <br /> <br /> {{location_logo_100}}</p>
+<p style="margin: 10px 0 0 0;"><a href="https://www.facebook.com/ArizonaComputerGuru"><img src="https://assets.services.syncromsp.com/assets/templates/email/icon-facebook-788cfb971ad9c52cacaf7ed02445ccbec753a3a3c6febc4240f55d778dddd819.png" alt="Facebook" /></a>&nbsp;<a href="https://twitter.com/azcomputerguru"><img src="https://assets.services.syncromsp.com/assets/templates/email/icon-twitter-7fe59727033d309e9ae65f2d13df61c3bc4bb367254c9d68d4d8c799cc4a0028.png" alt="Twitter" /></a></p>
+</td>
+</tr>
+
+<!-- FOOTER -->
+<tr>
+<td style="background-color: #1a1a1a; padding: 18px 35px; text-align: center;">
+<span style="font-size: 12px; color: #777777; font-family: Georgia, 'Times New Roman', serif;">Arizona Computer Guru &bull; 7437 E. 22nd St, Tucson, AZ 85710 &bull; 520.304.8300</span>
+</td>
+</tr>
+
+</table>
+<!-- END INNER CONTAINER -->
+
+</td>
+</tr>
+</table>
+<!-- END OUTER WRAPPER -->

projects/newsletter/variant-bold.html

@@ -0,0 +1,177 @@
+<!--[if mso]>
+<style>table, td {font-family: Arial, Helvetica, sans-serif !important;}</style>
+<![endif]-->
+
+<!-- VARIANT: BOLD / HIGH CONTRAST -->
+<table width="100%" cellpadding="0" cellspacing="0" border="0" style="background-color: #111111;">
+<tr>
+<td align="center" style="padding: 20px 10px;">
+
+<table width="640" cellpadding="0" cellspacing="0" border="0" style="font-family: Arial, Helvetica, sans-serif; color: #333333; line-height: 1.6;">
+
+<!-- HEADER -->
+<tr>
+<td style="background-color: #111111; padding: 25px 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td>
+<span style="font-size: 22px; font-weight: bold; color: #ffffff;">ARIZONA COMPUTER GURU</span>
+</td>
+<td align="right" style="vertical-align: middle;">
+<span style="font-size: 11px; color: #666666; letter-spacing: 2px;">TUCSON, AZ</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- FLASH SALE HERO -->
+<tr>
+<td style="background-color: #c0392b; padding: 40px 35px; text-align: center;">
+<span style="font-size: 12px; letter-spacing: 5px; color: #ffffff; text-transform: uppercase; font-weight: bold;">&#9889; One-Day Flash Sale &#9889;</span>
+<br /><br />
+<span style="font-size: 60px; font-weight: bold; color: #ffffff; line-height: 1.1;">$100<span style="font-size: 28px;">/hr</span></span>
+<br />
+<span style="font-size: 18px; color: #f5c6cb; text-decoration: line-through;">normally $150/hr</span>
+<br /><br />
+<table cellpadding="0" cellspacing="0" border="0" align="center">
+<tr>
+<td style="background-color: #ffffff; padding: 10px 25px;">
+<span style="font-size: 14px; font-weight: bold; color: #c0392b; letter-spacing: 1px;">SAVE $500 PER 10-HOUR BLOCK</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- WHITE CONTENT AREA -->
+<tr>
+<td style="background-color: #ffffff; padding: 30px 35px; font-family: Arial, Helvetica, sans-serif;">
+<p style="font-size: 16px; margin: 0 0 12px 0;">Hi there,</p>
+<p style="font-size: 16px; margin: 0 0 12px 0;">We don't do these often, so here's the short version.</p>
+<p style="font-size: 16px; margin: 0 0 20px 0;">For <strong>one day only</strong>, we're offering 10-hour prepaid labor blocks at <strong>$100/hour</strong>. That's <strong>$1,000 per block</strong> instead of the usual $1,500. If you've been putting off that server migration, network upgrade, or office buildout -- this is the time to lock in the hours.</p>
+</td>
+</tr>
+
+<!-- DETAILS STRIP -->
+<tr>
+<td style="background-color: #f2f2f2; padding: 25px 35px; font-family: Arial, Helvetica, sans-serif;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="font-size: 14px; color: #333; line-height: 2.0;">
+<strong style="font-size: 15px;">THE DETAILS:</strong><br />
+&#9654;&nbsp; 10-hour blocks at $100/hr (normally $150/hr)<br />
+&#9654;&nbsp; Limit: 4 blocks per client (40 hours max)<br />
+&#9654;&nbsp; Hours <strong>never expire</strong><br />
+&#9654;&nbsp; Use them for anything -- projects, support, on-site work<br />
+&#9654;&nbsp; <strong>One day only.</strong> When it's over, it's over.
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- CTA -->
+<tr>
+<td style="background-color: #ffffff; padding: 25px 35px;" align="center">
+<table cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="background-color: #c0392b; padding: 16px 50px; text-align: center;">
+<a href="tel:5203048300" style="font-family: Arial, Helvetica, sans-serif; font-size: 18px; font-weight: bold; color: #ffffff; text-decoration: none; letter-spacing: 0.5px;">CALL 520.304.8300</a>
+</td>
+</tr>
+</table>
+<p style="font-size: 13px; color: #888; margin: 10px 0 0 0; font-family: Arial, Helvetica, sans-serif;">or reply to this email &bull; first come, first served</p>
+</td>
+</tr>
+
+<!-- DARK SPACER -->
+<tr>
+<td style="background-color: #111111; height: 8px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+
+<!-- RADIO SHOW -->
+<tr>
+<td style="background-color: #ffffff; padding: 30px 35px; font-family: Arial, Helvetica, sans-serif;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="background-color: #c0392b; padding: 4px 12px; width: 1%; white-space: nowrap;">
+<span style="font-size: 10px; letter-spacing: 2px; color: #ffffff; text-transform: uppercase; font-weight: bold;">Returning</span>
+</td>
+<td>&nbsp;</td>
+</tr>
+</table>
+<p style="font-size: 22px; font-weight: bold; margin: 14px 0 10px 0; color: #111;">The Computer Guru Show</p>
+<p style="font-size: 15px; margin: 0 0 10px 0;">After a five-year break, we're bringing the radio show back. If you listened before, you know the format -- real talk about technology, security threats, and what actually matters for your business. No jargon, no fluff. New episodes are in production now for Season 11.</p>
+<p style="font-size: 15px; margin: 0;">194 classic episodes at <a href="https://radio.azcomputerguru.com" style="color: #c0392b; font-weight: bold;">radio.azcomputerguru.com</a></p>
+</td>
+</tr>
+
+<!-- DARK SPACER -->
+<tr>
+<td style="background-color: #111111; height: 8px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+
+<!-- SECURITY -->
+<tr>
+<td style="background-color: #ffffff; padding: 30px 35px; font-family: Arial, Helvetica, sans-serif;">
+<p style="font-size: 22px; font-weight: bold; margin: 0 0 10px 0; color: #111;">Are You Actually Protected?</p>
+<p style="font-size: 15px; margin: 0 0 14px 0;">Two things worth knowing about:</p>
+<p style="font-size: 15px; margin: 0 0 14px 0;"><strong>Penetration Testing.</strong> We'll attack your network the same way a real threat actor would -- then hand you a detailed report of what we found and how to fix it. Most businesses have no idea what's actually exposed until someone shows them. We're offering a <strong>free security risk assessment</strong> that includes an external vulnerability scan, dark web search for your company's compromised credentials, and a simulated phishing test on your staff. No obligation, no sales pitch -- just a written report with a risk score. Call us or reply to schedule yours.</p>
+<p style="font-size: 15px; margin: 0;"><strong>Advanced Antivirus for GPS Subscribers.</strong> If you're on our GPS-Pro or GPS-Advanced plans, you already have access to enterprise-grade EDR (Endpoint Detection &amp; Response) that goes well beyond what traditional antivirus catches. We're talking behavioral analysis, ransomware rollback, and real-time threat intelligence. If you're still on GPS-Basic and want to step up, now's a good time to talk about it.</p>
+</td>
+</tr>
+
+<!-- DARK SPACER -->
+<tr>
+<td style="background-color: #111111; height: 8px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+
+<!-- VOIP -->
+<tr>
+<td style="background-color: #ffffff; padding: 30px 35px; font-family: Arial, Helvetica, sans-serif;">
+<p style="font-size: 22px; font-weight: bold; margin: 0 0 10px 0; color: #111;">Still Overpaying for Phone Service?</p>
+<p style="font-size: 15px; margin: 0 0 14px 0;">Our ACG-Voice (powered by PacketDial) business phone system starts at <strong>$22/user/month</strong> with unlimited calling, voicemail-to-email, and a softphone app so your team can take calls from anywhere. The Standard tier at $28/user adds desk phone support, call queues, and ring groups -- everything a real office needs without the enterprise price tag.</p>
+<p style="font-size: 15px; margin: 0;">We handle the setup, port your existing numbers, and support the whole thing. If your current provider is nickel-and-diming you on features, let's have a conversation.</p>
+</td>
+</tr>
+
+<!-- DARK SPACER -->
+<tr>
+<td style="background-color: #111111; height: 8px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+
+<!-- WEB DEV -->
+<tr>
+<td style="background-color: #ffffff; padding: 30px 35px; font-family: Arial, Helvetica, sans-serif;">
+<p style="font-size: 22px; font-weight: bold; margin: 0 0 10px 0; color: #111;">Need a Website That Doesn't Embarrass You?</p>
+<p style="font-size: 15px; margin: 0;">We build and host business websites -- clean, fast, and actually maintained. Hosting starts at $15/month. If your site hasn't been touched since 2019 or you're paying a fortune for something that loads like it's on dial-up, we should talk. We also handle email hosting, SSL, and e-commerce if you need it.</p>
+</td>
+</tr>
+
+<!-- DARK SPACER -->
+<tr>
+<td style="background-color: #111111; height: 8px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+
+<!-- CLOSING + SIGNATURE -->
+<tr>
+<td style="background-color: #ffffff; padding: 30px 35px; font-family: Arial, Helvetica, sans-serif;">
+<p style="font-size: 16px; margin: 0 0 20px 0; color: #333;">We're grateful for you! Please reach out anytime -- we're here for you.</p>
+<p style="margin: 0 0 5px 0;"><span style="font-size: 20px;">Thanks,</span></p>
+<p style="margin: 0;"><span style="font-size: 20px;"> Michael Swanson<br />Owner<br /> </span><a href="http://www.azcomputerguru.com"><span style="font-size: 20px;">www.azcomputerguru.com</span></a> <span style="font-size: 20px;"><br /> phone:&nbsp;520.304.8300</span> <br /> <br /> {{location_logo_100}}</p>
+<p style="margin: 10px 0 0 0;"><a href="https://www.facebook.com/ArizonaComputerGuru"><img src="https://assets.services.syncromsp.com/assets/templates/email/icon-facebook-788cfb971ad9c52cacaf7ed02445ccbec753a3a3c6febc4240f55d778dddd819.png" alt="Facebook" /></a>&nbsp;<a href="https://twitter.com/azcomputerguru"><img src="https://assets.services.syncromsp.com/assets/templates/email/icon-twitter-7fe59727033d309e9ae65f2d13df61c3bc4bb367254c9d68d4d8c799cc4a0028.png" alt="Twitter" /></a></p>
+</td>
+</tr>
+
+<!-- FOOTER -->
+<tr>
+<td style="background-color: #111111; padding: 20px 35px; text-align: center;">
+<span style="font-size: 12px; color: #666; font-family: Arial, Helvetica, sans-serif;">Arizona Computer Guru &bull; 7437 E. 22nd St, Tucson, AZ 85710 &bull; 520.304.8300</span>
+</td>
+</tr>
+
+</table>
+</td>
+</tr>
+</table>

projects/newsletter/variant-clean.html

@@ -0,0 +1,199 @@
+<!--[if mso]>
+<style>table, td {font-family: Georgia, serif !important;}</style>
+<![endif]-->
+
+<!-- VARIANT: CLEAN / LIGHT -->
+<table width="100%" cellpadding="0" cellspacing="0" border="0" style="background-color: #ffffff;">
+<tr>
+<td align="center" style="padding: 20px 10px;">
+
+<table width="640" cellpadding="0" cellspacing="0" border="0" style="background-color: #ffffff; font-family: Georgia, 'Times New Roman', serif; color: #333333; line-height: 1.7;">
+
+<!-- HEADER -->
+<tr>
+<td style="padding: 30px 40px 20px 40px; border-bottom: 1px solid #e0e0e0;">
+<span style="font-size: 28px; font-weight: bold; color: #1a1a1a;">Arizona Computer Guru</span><br />
+<span style="font-size: 13px; color: #999; letter-spacing: 1.5px;">TUCSON IT &bull; SINCE 2001 &bull; 520.304.8300</span>
+</td>
+</tr>
+
+<!-- FLASH SALE -->
+<tr>
+<td style="padding: 35px 40px 0 40px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0" style="border: 2px solid #c0392b;">
+<tr>
+<td style="background-color: #c0392b; padding: 8px 20px;">
+<span style="font-size: 11px; letter-spacing: 3px; color: #ffffff; text-transform: uppercase; font-weight: bold;">One-Day Flash Sale</span>
+</td>
+</tr>
+<tr>
+<td style="padding: 30px 30px 25px 30px;">
+<span style="font-size: 36px; font-weight: bold; color: #c0392b;">$100/hr</span>
+<span style="font-size: 16px; color: #999; text-decoration: line-through; padding-left: 10px;">$150/hr</span><br />
+<span style="font-size: 15px; color: #555;">10-Hour Prepaid Labor Blocks &bull; Save $500 per block</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 25px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- BODY -->
+<tr>
+<td style="padding: 0 40px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 16px; margin: 0 0 14px 0;">Hi there,</p>
+<p style="font-size: 16px; margin: 0 0 14px 0;">We don't do these often, so here's the short version.</p>
+<p style="font-size: 16px; margin: 0 0 20px 0;">For <strong>one day only</strong>, we're offering 10-hour prepaid labor blocks at <strong>$100/hour</strong>. That's <strong>$1,000 per block</strong> instead of the usual $1,500. If you've been putting off that server migration, network upgrade, or office buildout -- this is the time to lock in the hours.</p>
+</td>
+</tr>
+
+<!-- DETAILS -->
+<tr>
+<td style="padding: 0 40px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="background-color: #f9f9f9; padding: 20px 25px; border-left: 3px solid #c0392b; font-family: Georgia, 'Times New Roman', serif;">
+<span style="font-size: 14px; color: #333; line-height: 1.9;">
+&#8226;&nbsp; 10-hour blocks at $100/hr (normally $150/hr)<br />
+&#8226;&nbsp; Limit: 4 blocks per client (40 hours max)<br />
+&#8226;&nbsp; Hours <strong>never expire</strong><br />
+&#8226;&nbsp; Use them for anything -- projects, support, on-site work<br />
+&#8226;&nbsp; <strong>One day only.</strong> When it's over, it's over.
+</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 20px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- CTA -->
+<tr>
+<td align="center" style="padding: 0 40px;">
+<table cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="border: 2px solid #c0392b; padding: 12px 35px; text-align: center;">
+<a href="tel:5203048300" style="font-family: Georgia, 'Times New Roman', serif; font-size: 16px; font-weight: bold; color: #c0392b; text-decoration: none;">Call 520.304.8300 to Claim Yours</a>
+</td>
+</tr>
+</table>
+<p style="font-size: 13px; color: #999; margin: 8px 0 0 0; font-family: Georgia, 'Times New Roman', serif;">or reply to this email &bull; first come, first served</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 30px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- DIVIDER -->
+<tr><td style="padding: 0 40px;"><table width="100%" cellpadding="0" cellspacing="0" border="0"><tr><td style="border-top: 1px solid #e0e0e0; height: 1px; font-size: 1px;">&nbsp;</td></tr></table></td></tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- RADIO SHOW -->
+<tr>
+<td style="padding: 0 40px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 11px; letter-spacing: 2px; color: #c0392b; text-transform: uppercase; font-weight: bold; margin: 0 0 6px 0;">Now Returning</p>
+<p style="font-size: 20px; font-weight: bold; margin: 0 0 10px 0; color: #1a1a1a;">The Computer Guru Show</p>
+<p style="font-size: 15px; margin: 0 0 10px 0;">After a five-year break, we're bringing the radio show back. If you listened before, you know the format -- real talk about technology, security threats, and what actually matters for your business. No jargon, no fluff. New episodes are in production now for Season 11.</p>
+<p style="font-size: 15px; margin: 0;">194 classic episodes are already archived at <a href="https://radio.azcomputerguru.com" style="color: #c0392b;">radio.azcomputerguru.com</a></p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- DIVIDER -->
+<tr><td style="padding: 0 40px;"><table width="100%" cellpadding="0" cellspacing="0" border="0"><tr><td style="border-top: 1px solid #e0e0e0; height: 1px; font-size: 1px;">&nbsp;</td></tr></table></td></tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- SECURITY -->
+<tr>
+<td style="padding: 0 40px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 20px; font-weight: bold; margin: 0 0 10px 0; color: #1a1a1a;">Are You Actually Protected?</p>
+<p style="font-size: 15px; margin: 0 0 14px 0;">Two things worth knowing about:</p>
+<p style="font-size: 15px; margin: 0 0 14px 0;"><strong>Penetration Testing.</strong> We'll attack your network the same way a real threat actor would -- then hand you a detailed report of what we found and how to fix it. Most businesses have no idea what's actually exposed until someone shows them. We're offering a <strong>free security risk assessment</strong> that includes an external vulnerability scan, dark web search for your company's compromised credentials, and a simulated phishing test on your staff. No obligation, no sales pitch -- just a written report with a risk score. Call us or reply to schedule yours.</p>
+<p style="font-size: 15px; margin: 0 0 5px 0;"><strong>Advanced Antivirus for GPS Subscribers.</strong> If you're on our GPS-Pro or GPS-Advanced plans, you already have access to enterprise-grade EDR (Endpoint Detection &amp; Response) that goes well beyond what traditional antivirus catches. We're talking behavioral analysis, ransomware rollback, and real-time threat intelligence. If you're still on GPS-Basic and want to step up, now's a good time to talk about it.</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- DIVIDER -->
+<tr><td style="padding: 0 40px;"><table width="100%" cellpadding="0" cellspacing="0" border="0"><tr><td style="border-top: 1px solid #e0e0e0; height: 1px; font-size: 1px;">&nbsp;</td></tr></table></td></tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- VOIP -->
+<tr>
+<td style="padding: 0 40px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 20px; font-weight: bold; margin: 0 0 10px 0; color: #1a1a1a;">Still Overpaying for Phone Service?</p>
+<p style="font-size: 15px; margin: 0 0 14px 0;">Our ACG-Voice (powered by PacketDial) business phone system starts at <strong>$22/user/month</strong> with unlimited calling, voicemail-to-email, and a softphone app so your team can take calls from anywhere. The Standard tier at $28/user adds desk phone support, call queues, and ring groups -- everything a real office needs without the enterprise price tag.</p>
+<p style="font-size: 15px; margin: 0 0 5px 0;">We handle the setup, port your existing numbers, and support the whole thing. If your current provider is nickel-and-diming you on features, let's have a conversation.</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- DIVIDER -->
+<tr><td style="padding: 0 40px;"><table width="100%" cellpadding="0" cellspacing="0" border="0"><tr><td style="border-top: 1px solid #e0e0e0; height: 1px; font-size: 1px;">&nbsp;</td></tr></table></td></tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- WEB DEV -->
+<tr>
+<td style="padding: 0 40px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 20px; font-weight: bold; margin: 0 0 10px 0; color: #1a1a1a;">Need a Website That Doesn't Embarrass You?</p>
+<p style="font-size: 15px; margin: 0 0 5px 0;">We build and host business websites -- clean, fast, and actually maintained. Hosting starts at $15/month. If your site hasn't been touched since 2019 or you're paying a fortune for something that loads like it's on dial-up, we should talk. We also handle email hosting, SSL, and e-commerce if you need it.</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 28px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- DIVIDER -->
+<tr><td style="padding: 0 40px;"><table width="100%" cellpadding="0" cellspacing="0" border="0"><tr><td style="border-top: 1px solid #e0e0e0; height: 1px; font-size: 1px;">&nbsp;</td></tr></table></td></tr>
+
+<!-- SPACER -->
+<tr><td style="height: 25px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- CLOSING -->
+<tr>
+<td style="padding: 0 40px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 16px; margin: 0 0 5px 0; color: #333;">We're grateful for you! Please reach out anytime -- we're here for you.</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="height: 25px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- SIGNATURE -->
+<tr>
+<td style="padding: 0 40px 30px 40px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="margin: 0 0 5px 0;"><span style="font-size: 20px;">Thanks,</span></p>
+<p style="margin: 0;"><span style="font-size: 20px;"> Michael Swanson<br />Owner<br /> </span><a href="http://www.azcomputerguru.com"><span style="font-size: 20px;">www.azcomputerguru.com</span></a> <span style="font-size: 20px;"><br /> phone:&nbsp;520.304.8300</span> <br /> <br /> {{location_logo_100}}</p>
+<p style="margin: 10px 0 0 0;"><a href="https://www.facebook.com/ArizonaComputerGuru"><img src="https://assets.services.syncromsp.com/assets/templates/email/icon-facebook-788cfb971ad9c52cacaf7ed02445ccbec753a3a3c6febc4240f55d778dddd819.png" alt="Facebook" /></a>&nbsp;<a href="https://twitter.com/azcomputerguru"><img src="https://assets.services.syncromsp.com/assets/templates/email/icon-twitter-7fe59727033d309e9ae65f2d13df61c3bc4bb367254c9d68d4d8c799cc4a0028.png" alt="Twitter" /></a></p>
+</td>
+</tr>
+
+<!-- FOOTER -->
+<tr>
+<td style="padding: 15px 40px; border-top: 1px solid #e0e0e0; text-align: center;">
+<span style="font-size: 12px; color: #999; font-family: Georgia, 'Times New Roman', serif;">Arizona Computer Guru &bull; 7437 E. 22nd St, Tucson, AZ 85710 &bull; 520.304.8300</span>
+</td>
+</tr>
+
+</table>
+</td>
+</tr>
+</table>

projects/newsletter/variant-warm.html

@@ -0,0 +1,233 @@
+<!--[if mso]>
+<style>table, td {font-family: Georgia, serif !important;}</style>
+<![endif]-->
+
+<!-- VARIANT: WARM / FRIENDLY -->
+<table width="100%" cellpadding="0" cellspacing="0" border="0" style="background-color: #e8e0d4;">
+<tr>
+<td align="center" style="padding: 20px 10px;">
+
+<table width="640" cellpadding="0" cellspacing="0" border="0" style="font-family: Georgia, 'Times New Roman', serif; color: #3a3530; line-height: 1.7;">
+
+<!-- HEADER -->
+<tr>
+<td style="background-color: #3a3530; padding: 20px 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td>
+<span style="font-size: 24px; font-weight: bold; color: #f0e8dc;">Arizona Computer Guru</span>
+</td>
+</tr>
+<tr>
+<td style="padding-top: 4px;">
+<span style="font-size: 12px; color: #a09484; letter-spacing: 1px;">Your Tucson IT team since 2001</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- WARM ACCENT -->
+<tr>
+<td style="background-color: #c45c3c; height: 5px; font-size: 5px; line-height: 5px;">&nbsp;</td>
+</tr>
+
+<!-- FLASH SALE -->
+<tr>
+<td style="background-color: #faf6f0; padding: 30px 35px; text-align: center;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td align="center">
+<span style="font-size: 12px; letter-spacing: 3px; color: #c45c3c; text-transform: uppercase; font-weight: bold;">Flash Sale &bull; Thursday, April 9th &bull; 9AM - 5PM</span>
+</td>
+</tr>
+<tr>
+<td align="center" style="padding-top: 15px;">
+<table cellpadding="0" cellspacing="0" border="0" align="center">
+<tr>
+<td style="background-color: #3a3530; padding: 20px 40px; text-align: center;">
+<span style="font-size: 14px; color: #a09484;">Prepaid Labor Blocks</span><br />
+<span style="font-size: 38px; font-weight: bold; color: #f0e8dc;">$100/hr</span><br />
+<span style="font-size: 14px; color: #a09484; text-decoration: line-through;">$150/hr regular rate</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+<tr>
+<td align="center" style="padding-top: 12px;">
+<span style="font-size: 15px; color: #7a6e60;">10-hour blocks &bull; save $500 each &bull; limit 4 per client</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- BODY -->
+<tr>
+<td style="background-color: #faf6f0; padding: 10px 35px 25px 35px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 16px; margin: 0 0 14px 0; color: #3a3530;">Hi there,</p>
+<p style="font-size: 16px; margin: 0 0 14px 0; color: #3a3530;">We don't do these often, so here's the short version.</p>
+<p style="font-size: 16px; margin: 0 0 14px 0; color: #3a3530;">For <strong>today only (Thursday, April 9th, 9AM - 5PM)</strong>, we're offering 10-hour prepaid labor blocks at <strong>$100/hour</strong>. That's <strong>$1,000 per block</strong> instead of the usual $1,500. If you've been putting off that server migration, network upgrade, or office buildout -- or if you just want to lock in a considerably lower hourly rate for whenever you need us -- this is your opportunity.</p>
+<p style="font-size: 16px; margin: 0 0 20px 0; color: #3a3530;">And if you've been paying our standard rate of $175/hr on an as-needed basis, this is worth a serious look. You'd be cutting your hourly cost nearly in half, and the hours never expire. Use them whenever you need us -- no rush, no deadline.</p>
+</td>
+</tr>
+
+<!-- DETAILS -->
+<tr>
+<td style="background-color: #faf6f0; padding: 0 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="background-color: #f0e8dc; padding: 20px 25px; border-left: 4px solid #c45c3c; font-family: Georgia, 'Times New Roman', serif;">
+<span style="font-size: 15px; font-weight: bold; color: #3a3530;">The Details:</span><br />
+<span style="font-size: 14px; color: #3a3530; line-height: 1.9;">
+&#8226;&nbsp; 10-hour blocks at $100/hr (normally $150/hr)<br />
+&#8226;&nbsp; Limit: 4 blocks per client (40 hours max)<br />
+&#8226;&nbsp; Hours <strong>never expire</strong><br />
+&#8226;&nbsp; Use them for anything -- projects, support, on-site work<br />
+&#8226;&nbsp; <strong>Today only, 9AM - 5PM.</strong> When it's over, it's over.
+</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="background-color: #faf6f0; height: 22px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- CTA -->
+<tr>
+<td style="background-color: #faf6f0; padding: 0 35px;" align="center">
+<table cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="background-color: #c45c3c; padding: 14px 40px; text-align: center;">
+<a href="tel:5203048300" style="font-family: Georgia, 'Times New Roman', serif; font-size: 17px; font-weight: bold; color: #ffffff; text-decoration: none;">Call 520.304.8300 to Claim Yours</a>
+</td>
+</tr>
+</table>
+<p style="font-size: 13px; color: #a09484; margin: 8px 0 0 0; font-family: Georgia, 'Times New Roman', serif;">or reply to this email &bull; first come, first served</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr><td style="background-color: #faf6f0; height: 30px; font-size: 1px; line-height: 1px;">&nbsp;</td></tr>
+
+<!-- TRANSITION -->
+<tr>
+<td style="background-color: #e8e0d4; height: 12px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+
+<!-- SECURITY BANNER -->
+<tr>
+<td style="background-color: #3a3530; padding: 16px 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="border-left: 4px solid #c45c3c; padding-left: 15px;">
+<span style="font-size: 11px; letter-spacing: 3px; color: #a09484; text-transform: uppercase;">Cybersecurity</span><br />
+<span style="font-size: 20px; font-weight: bold; color: #f0e8dc;">Are You Actually Protected?</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- SECURITY CONTENT -->
+<tr>
+<td style="background-color: #ffffff; padding: 28px 35px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 15px; margin: 0 0 14px 0; color: #3a3530;">Here's something we're seeing more and more: attacks that are written by AI. Not the clumsy phishing emails full of typos that you could spot from a mile away. These are polished, personalized, and they're getting past people who know better. AI has made it cheap and easy for attackers to scale up -- and small businesses are squarely in the crosshairs because most of them aren't prepared for it.</p>
+
+<p style="font-size: 15px; margin: 0 0 14px 0; color: #3a3530;">That's why we've been pushing two things lately:</p>
+
+<p style="font-size: 15px; margin: 0 0 14px 0; color: #3a3530;"><strong>Penetration Testing.</strong> We go after your network the same way an actual attacker would -- probing for weak spots, testing your defenses, seeing what's exposed. Then we hand you a plain-English report that tells you exactly what we found and what to do about it. Most businesses are genuinely surprised by the results. Right now we're offering a <strong>free security risk assessment</strong> -- that includes an external vulnerability scan, a dark web search for your company's compromised credentials, and a simulated phishing test on your staff. No strings, no sales pitch. Just a written report with a risk score. If you're curious, just reply or give us a call.</p>
+
+<p style="font-size: 15px; margin: 0; color: #3a3530;"><strong>Advanced Antivirus for GPS Subscribers.</strong> If you're on our GPS-Pro or GPS-Advanced plans, you've already got enterprise-grade EDR running on your machines -- that's Endpoint Detection &amp; Response, and it catches the stuff that traditional antivirus flat-out misses. Think behavioral analysis, ransomware rollback, real-time threat intelligence. It's the kind of protection that used to be reserved for big companies with big budgets. If you're still on GPS-Basic, let's talk about whether it makes sense to step up. The threat landscape has changed a lot, even in the last year.</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr>
+<td style="background-color: #e8e0d4; height: 8px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+
+<!-- VOIP BANNER -->
+<tr>
+<td style="background-color: #3a3530; padding: 16px 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="border-left: 4px solid #c45c3c; padding-left: 15px;">
+<span style="font-size: 11px; letter-spacing: 3px; color: #a09484; text-transform: uppercase;">Business Phone Systems</span><br />
+<span style="font-size: 20px; font-weight: bold; color: #f0e8dc;">Still Overpaying for Phone Service?</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- VOIP CONTENT -->
+<tr>
+<td style="background-color: #ffffff; padding: 28px 35px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 15px; margin: 0 0 14px 0; color: #3a3530;">We talk to business owners all the time who are paying way too much for phone service that doesn't even do what they need. Locked into contracts, paying per-feature fees, dealing with clunky hardware from 2015. Sound familiar?</p>
+
+<p style="font-size: 15px; margin: 0 0 14px 0; color: #3a3530;">Our ACG-Voice system (powered by PacketDial) starts at <strong>$22/user/month</strong>. That gets you unlimited calling, voicemail-to-email, and a softphone app so your team can take calls from their desk, their laptop, or their phone -- wherever they are. The Standard tier at $28/user adds desk phone support, call queues, and ring groups. Basically everything a real office needs without the enterprise price tag.</p>
+
+<p style="font-size: 15px; margin: 0; color: #3a3530;">We handle the setup, port your existing numbers over, and support the whole thing going forward. If your current provider has been nickel-and-diming you, let's have a conversation. It usually takes about five minutes to figure out if we can save you money.</p>
+</td>
+</tr>
+
+<!-- SPACER -->
+<tr>
+<td style="background-color: #e8e0d4; height: 8px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+
+<!-- RADIO SHOW BANNER -->
+<tr>
+<td style="background-color: #3a3530; padding: 16px 35px;">
+<table width="100%" cellpadding="0" cellspacing="0" border="0">
+<tr>
+<td style="border-left: 4px solid #c45c3c; padding-left: 15px;">
+<span style="font-size: 11px; letter-spacing: 3px; color: #c45c3c; text-transform: uppercase;">Now Returning</span><br />
+<span style="font-size: 20px; font-weight: bold; color: #f0e8dc;">The Computer Guru Show</span>
+</td>
+</tr>
+</table>
+</td>
+</tr>
+
+<!-- RADIO SHOW CONTENT -->
+<tr>
+<td style="background-color: #ffffff; padding: 28px 35px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 15px; margin: 0 0 14px 0; color: #3a3530;">Some of you might remember our radio show -- we ran it for years before taking a break back in 2018. Well, it's coming back. We've been working on new episodes for Season 11, and honestly, there's never been a better time. Between AI, ransomware, and everything happening in the tech world right now, there's no shortage of things to talk about.</p>
+
+<p style="font-size: 15px; margin: 0 0 14px 0; color: #3a3530;">The format is the same as it always was: straight talk about technology, security, and what it all means for regular people running businesses. No buzzwords, no sponsored segments, just us breaking it down.</p>
+
+<p style="font-size: 15px; margin: 0; color: #3a3530;">In the meantime, all 194 classic episodes are archived and ready to listen at <a href="https://radio.azcomputerguru.com" style="color: #c45c3c; font-weight: bold;">radio.azcomputerguru.com</a>. We'll let you know when the new season drops.</p>
+</td>
+</tr>
+
+<!-- BOTTOM SPACER -->
+<tr>
+<td style="background-color: #e8e0d4; height: 3px; font-size: 1px; line-height: 1px;">&nbsp;</td>
+</tr>
+
+<!-- CLOSING + SIGNATURE -->
+<tr>
+<td style="background-color: #ffffff; padding: 28px 35px; font-family: Georgia, 'Times New Roman', serif;">
+<p style="font-size: 16px; margin: 0 0 20px 0; color: #3a3530;">We're grateful for you! Please reach out anytime -- we're here for you.</p>
+<p style="margin: 0 0 5px 0;"><span style="font-size: 20px;">Thanks,</span></p>
+<p style="margin: 0;"><span style="font-size: 20px;"> Michael Swanson<br />Owner<br /> </span><a href="http://www.azcomputerguru.com"><span style="font-size: 20px;">www.azcomputerguru.com</span></a> <span style="font-size: 20px;"><br /> phone:&nbsp;520.304.8300</span> <br /> <br /> {{location_logo_100}}</p>
+<p style="margin: 10px 0 0 0;"><a href="https://www.facebook.com/ArizonaComputerGuru"><img src="https://assets.services.syncromsp.com/assets/templates/email/icon-facebook-788cfb971ad9c52cacaf7ed02445ccbec753a3a3c6febc4240f55d778dddd819.png" alt="Facebook" /></a>&nbsp;<a href="https://twitter.com/azcomputerguru"><img src="https://assets.services.syncromsp.com/assets/templates/email/icon-twitter-7fe59727033d309e9ae65f2d13df61c3bc4bb367254c9d68d4d8c799cc4a0028.png" alt="Twitter" /></a></p>
+</td>
+</tr>
+
+<!-- FOOTER -->
+<tr>
+<td style="background-color: #3a3530; padding: 18px 35px; text-align: center;">
+<span style="font-size: 12px; color: #a09484; font-family: Georgia, 'Times New Roman', serif;">Arizona Computer Guru &bull; 7437 E. 22nd St, Tucson, AZ 85710 &bull; 520.304.8300</span>
+</td>
+</tr>
+
+</table>
+</td>
+</tr>
+</table>