TraceEvent Importer (managed) replacement for ReadTrace

[PiJoCoder]

Introduction

Trace Event Importer is a fully managed SqlNexus importer plugin that reads SQL Server trace files (.trc and .xel) directly in C#, eliminating the dependency on the legacy native ReadTrace.exe executable. It normalizes SQL text, computes hash-based batch/statement groupings, and bulk-loads processed data into the existing ReadTrace database schema used by SqlNexus for workload analysis. It creates identical database schema and objects as those of ReadTrace to avoid having to recreate reports and queries.

Key Design Decisions

1. Same schema, same reports: The importer writes to the identical ReadTrace.* tables that ReadTrace.exe creates. All existing RDLC reports work without modification.

2. Embedded SQL scripts: CreateSchema.sql and PostLoadFixups.sql are embedded resources in the DLL, eliminating external file dependencies.

3. INexusImporter (not INexusFileImporter): The plugin handles its own file enumeration internally (same pattern as ReadTrace), receiving a file mask from the host and processing all matching files in a single DoImport() call.

4. XELite for XEL reading: Uses the Microsoft.SqlServer.XEvent.XELite NuGet package for reading Extended Events files, which handles the binary XEL format reliably.

5. BulkLoadRowset for writing: Uses the existing BulkLoadEx library (shared with other importers) for efficient bulk insert.

Implementation Plan

Phase 1: Core Pipeline

• Event model (TraceEvent.cs): Define a unified event structure that can represent both TRC and XEL events with fields for session, request, timestamps, duration, CPU, reads, writes, text data, etc.
• TRC file reader (TrcFileReader.cs): Read SQL Trace binary format, map trace event IDs to TraceEventType enum, extract fields by column ID.
• XEL file reader (XelFileReader.cs): Read Extended Events files via XELite, map XE event names (sql_batch_completed, rpc_completed, etc.) to the same TraceEventType enum.
• Event processor (EventProcessor.cs): Correlate Starting↔Completed events per session+request, track connections (login→logout), handle attention events.

Phase 2: Normalization & Hashing

• SQL text normalizer (SqlTextNormalizer.cs): Replace string literals with {STR}, numeric literals with {##}, normalize whitespace, handle quoted identifiers.
• Hash computation (HashComputer.cs): Compute stable 64-bit hash IDs for normalized SQL text (must produce consistent hashes for grouping).
• Special procedure detection (SpecialProcDetector.cs): Identify sp_executesql, sp_prepare, sp_prepexec, sp_cursoropen, etc. and extract the inner SQL text for normalization.
• Unique store (UniqueStore.cs): Deduplicate batches, statements, app names, login names, procedure names; assign sequential IDs.

Phase 3: Aggregation & Database Writing

• Time-interval aggregation (Aggregator.cs): Partition events into configurable time intervals (default 60 seconds), compute per-HashID partial aggregates (min/max/total for duration, reads, writes, CPU; counts for starting/completed/attention events).
• Bulk writer (BulkWriter.cs): Write all ReadTrace tables via BulkLoadRowset — metadata, reference data, unique text, fact tables, and aggregation tables.
• Schema deployment (CreateSchema.sql): CREATE SCHEMA, all tables matching the ReadTrace.exe layout, and the vwBatchPartialAggsByGroupTimeInterval view.
• Post-load fixups (PostLoadFixups.sql): Create primary keys, nonclustered indexes (using ReadTrace.exe naming convention for index hint compatibility), and reconcile ConnSeq/BatchSeq/ParentStmtSeq foreign key relationships.

Phase 4: Host Integration

• Plugin registration: Add ProjectReference in sqlnexus.csproj, register in AppConfig.xml.
• Mutual exclusivity with ReadTrace: Since both importers write to the same ReadTrace schema, only one can be active at a time.
  • Auto-disable ReadTrace when TraceEventImporter is enabled (and vice versa) at startup
  • Show a MessageBox informing the user when toggling between importers at runtime
  • Persist the selection via SaveImportOptions
• File mask alignment: Use *pssdiag*.xel and *LogScout*.xel masks (not *.xel) to avoid picking up system health, AlwaysOn, or SQLDiag XEL files that belong to the CustomXELImporter.
• Post-processing script: Declare ReadTracePostProcessing.sql as a post-script, and update the host's RunPostScripts to allow it for both ReadTrace and TraceEventImporter.
• Non-boolean options: Handle non-boolean options like "Aggregation interval (seconds)" gracefully in the UI (skip from checkbox menu).
• New importer defaults: Ensure new importers default to their constructor-defined values on first run (not false) when no saved options exist yet.

Phase 5: Schema Compatibility

The following objects must match what ReadTracePostProcessing.sql and the RDLC reports expect:

• View: ReadTrace.vwBatchPartialAggsByGroupTimeInterval — must return exactly 10 columns (StartTime, EndTime, TimeInterval, StartingEvents, CompletedEvents, Attentions, Duration, Reads, Writes, CPU) aggregated across all HashIDs per time interval.
• Index names: Must use the ReadTrace.exe naming convention (e.g., tblBatches_HashID not IX_tblBatches_HashID) because ReadTracePostProcessing.sql uses WITH (INDEX(...)) hints that reference these exact names.
• Column types and names: All table columns must match the types and names that ReadTracePostProcessing.sql stored procedures and the RDLC report datasets expect.

Phase 6: BulkLoadRowset Fix

• Schema-qualified table names: BulkLoadRowset.GetNewRow() wraps the table name in [brackets], which breaks for schema-qualified names like ReadTrace.tblBatches (produces [ReadTrace.tblBatches] instead of [ReadTrace].[tblBatches]). Fix to split on . and bracket each part separately.

Configuration

Option	Type	Default	Description
Enabled	bool	true	Enable/disable the importer
Drop existing ReadTrace tables	bool	true	Drop and recreate schema before import
Aggregation interval (seconds)	int	60	Time bucket size for partial aggregations

Testing Plan

1. Build & Deploy Verification

• Clean and rebuild the solution
• Confirm TraceEventImporter.dll is present in sqlnexus\bin\Release\
• Confirm all embedded resources are included (CreateSchema.sql, PostLoadFixups.sql)

2. Importer Discovery & UI

• Launch SQL Nexus, open the Import form
• Verify "Trace Event Importer (Managed)" appears under Options → Importers with Enabled checked
• Verify "ReadTrace (SQL XEL/TRC Files)" is auto-disabled when TraceEventImporter is enabled
• Toggle between the two importers — verify the MessageBox notification appears and the other is disabled
• Verify the "Aggregation interval (seconds)" option does NOT appear as a checkbox (non-boolean options are hidden)

3. Basic Import (XEL files)

• Point to a SQL LogScout output folder containing *LogScout*.xel files
• Click Import. Verify:
  • The XEL file(s) appear in the file list with progress bars
  • The ReadTrace schema and all tables are created
  • All indexes exist with the correct names (no IX_ prefix)
  • ReadTrace.vwBatchPartialAggsByGroupTimeInterval view exists
  • Data is populated in tblBatches, tblStatements, tblConnections, tblUniqueBatches, etc.

4. Basic Import (TRC files)

• Point to a PSSDIAG output folder containing *sp_trace*.trc files
• Verify the same successful import flow as with XEL files

5. Report Validation

• After import, open each ReadTrace report and verify no errors:
  • ReadTrace Main — time-series charts for batches, CPU, duration, reads, writes
  • Top Unique Batches — top-N table with clickable query templates
  • Batch Details (click a query template) — verify no index hint errors
  • Top Unique Statements — similar drill-down
  • Interesting Events — recompiles, autogrow, etc.
  • ReadTrace Warnings — any import warnings

6. Comparative Testing (TraceEventImporter vs ReadTrace)

This is the critical validation — import the same trace files with both importers and compare results.

Setup

1. Collect a representative set of trace files (both .xel and .trc if possible)
2. Create two separate databases (e.g., sqlnexus_managed and sqlnexus_readtrace)
3. Import with TraceEventImporter into sqlnexus_managed
4. Import with ReadTrace (SQL XEL/TRC Files) into sqlnexus_readtrace

Comparison Queries

-- Compare total event counts 
SELECT 'Managed' AS Source, COUNT(*) AS BatchCount 
FROM [sqlnexus_managed].ReadTrace.tblBatches 
UNION ALL 
SELECT 'ReadTrace', COUNT(*) 
FROM [sqlnexus_readtrace].ReadTrace.tblBatches

-- Compare unique batch counts 
SELECT 'Managed' AS Source, COUNT(*) AS UniqueBatches 
FROM [sqlnexus_managed].ReadTrace.tblUniqueBatches 
UNION ALL 
SELECT 'ReadTrace', COUNT(*) 
FROM [sqlnexus_readtrace].ReadTrace.tblUniqueBatches

-- Compare statement counts 
SELECT 'Managed' AS Source, COUNT(*) AS StmtCount 
FROM [sqlnexus_managed].ReadTrace.tblStatements 
UNION ALL 
SELECT 'ReadTrace', COUNT(*) 
FROM [sqlnexus_readtrace].ReadTrace.tblStatements

-- Compare connection counts 
SELECT 'Managed' AS Source, COUNT(*) AS ConnCount 
FROM [sqlnexus_managed].ReadTrace.tblConnections 
UNION ALL 
SELECT 'ReadTrace', COUNT(*) 
FROM [sqlnexus_readtrace].ReadTrace.tblConnections

-- Compare top-10 batches by total CPU (should be similar ranking) 
SELECT TOP 10 ub.NormText, SUM(b.CPU) AS TotalCPU, COUNT(*) AS Executes 
FROM [sqlnexus_managed].ReadTrace.tblBatches b 
  JOIN [sqlnexus_managed].ReadTrace.tblUniqueBatches ub 
  ON b.HashID = ub.HashID WHERE b.CPU IS NOT NULL 
GROUP BY ub.NormText 
ORDER BY TotalCPU DESC

-- Same query against ReadTrace database for comparison 
SELECT TOP 10 ub.NormText, SUM(b.CPU) AS TotalCPU, COUNT(*) AS Executes 
FROM [sqlnexus_readtrace].ReadTrace.tblBatches b 
  JOIN [sqlnexus_readtrace].ReadTrace.tblUniqueBatches ub 
  ON b.HashID = ub.HashID WHERE b.CPU IS NOT NULL 
GROUP BY ub.NormText 
ORDER BY TotalCPU DESC

-- Compare time interval aggregation totals 
SELECT 'Managed' AS Source, COUNT(*) AS Intervals, SUM(StartingEvents) AS TotalStarting, SUM(CompletedEvents) AS TotalCompleted 
FROM [sqlnexus_managed].ReadTrace.vwBatchPartialAggsByGroupTimeInterval 
UNION ALL 
SELECT 'ReadTrace', COUNT(*), SUM(StartingEvents), SUM(CompletedEvents) 
FROM [sqlnexus_readtrace].ReadTrace.vwBatchPartialAggsByGroupTimeInterval

More test queries:

-- 1. HashID overlap: how many unique batches exist in both, only Managed, only ReadTrace
SELECT
    SUM(CASE WHEN m.HashID IS NOT NULL AND r.HashID IS NOT NULL THEN 1 ELSE 0 END) AS InBoth,
    SUM(CASE WHEN m.HashID IS NOT NULL AND r.HashID IS NULL     THEN 1 ELSE 0 END) AS ManagedOnly,
    SUM(CASE WHEN m.HashID IS NULL     AND r.HashID IS NOT NULL THEN 1 ELSE 0 END) AS ReadTraceOnly
FROM      [sqlnexus_managed].ReadTrace.tblUniqueBatches   m
FULL JOIN [sqlnexus_readtrace].ReadTrace.tblUniqueBatches r ON m.HashID = r.HashID;

-- 2. Per-HashID execution count agreement (rows where counts differ by more than 1%)
SELECT
    m.HashID,
    m.ExecCount  AS Managed_Execs,
    r.ExecCount  AS ReadTrace_Execs,
    ABS(m.ExecCount - r.ExecCount) AS Diff
FROM (
    SELECT HashID, COUNT(*) AS ExecCount
    FROM [sqlnexus_managed].ReadTrace.tblBatches
    GROUP BY HashID
) m
JOIN (
    SELECT HashID, COUNT(*) AS ExecCount
    FROM [sqlnexus_readtrace].ReadTrace.tblBatches
    GROUP BY HashID
) r ON m.HashID = r.HashID
WHERE ABS(m.ExecCount - r.ExecCount) > 0
ORDER BY Diff DESC;

-- 3. Aggregate CPU/Reads/Writes/Duration totals across all batches
SELECT 'Managed'   AS Source, SUM(CPU) AS TotalCPU, SUM(Reads) AS TotalReads,
       SUM(Writes) AS TotalWrites, SUM(Duration) AS TotalDuration
FROM [sqlnexus_managed].ReadTrace.tblBatches
UNION ALL
SELECT 'ReadTrace', SUM(CPU), SUM(Reads), SUM(Writes), SUM(Duration)
FROM [sqlnexus_readtrace].ReadTrace.tblBatches;

-- 4. Batch CPU/Reads/Duration agreement per HashID (top 20 diverging)
SELECT
    m.HashID,
    m.TotalCPU    AS M_CPU,    r.TotalCPU    AS RT_CPU,
    m.TotalReads  AS M_Reads,  r.TotalReads  AS RT_Reads,
    m.TotalDur    AS M_Dur,    r.TotalDur    AS RT_Dur
FROM (
    SELECT HashID,
           SUM(CPU)      AS TotalCPU,
           SUM(Reads)    AS TotalReads,
           SUM(Duration) AS TotalDur
    FROM [sqlnexus_managed].ReadTrace.tblBatches
    GROUP BY HashID
) m
JOIN (
    SELECT HashID,
           SUM(CPU)      AS TotalCPU,
           SUM(Reads)    AS TotalReads,
           SUM(Duration) AS TotalDur
    FROM [sqlnexus_readtrace].ReadTrace.tblBatches
    GROUP BY HashID
) r ON m.HashID = r.HashID
ORDER BY ABS(ISNULL(m.TotalCPU,0) - ISNULL(r.TotalCPU,0)) DESC
OFFSET 0 ROWS FETCH NEXT 20 ROWS ONLY;

-- 5. NormText agreement for matching HashIDs (any that differ indicate a normalization bug)
SELECT
    m.HashID,
    m.NormText AS Managed_NormText,
    r.NormText AS ReadTrace_NormText
FROM [sqlnexus_managed].ReadTrace.tblUniqueBatches   m
JOIN [sqlnexus_readtrace].ReadTrace.tblUniqueBatches r ON m.HashID = r.HashID
WHERE m.NormText <> r.NormText;

-- 6. SpecialProcID agreement for matching HashIDs
SELECT
    m.HashID,
    m.SpecialProcID AS Managed_SpecialProcID,
    r.SpecialProcID AS ReadTrace_SpecialProcID
FROM [sqlnexus_managed].ReadTrace.tblUniqueBatches   m
JOIN [sqlnexus_readtrace].ReadTrace.tblUniqueBatches r ON m.HashID = r.HashID
WHERE m.SpecialProcID <> r.SpecialProcID;

-- 7. Statement HashID overlap (same concept as query 1 but for tblUniqueStatements)
SELECT
    SUM(CASE WHEN m.HashID IS NOT NULL AND r.HashID IS NOT NULL THEN 1 ELSE 0 END) AS InBoth,
    SUM(CASE WHEN m.HashID IS NOT NULL AND r.HashID IS NULL     THEN 1 ELSE 0 END) AS ManagedOnly,
    SUM(CASE WHEN m.HashID IS NULL     AND r.HashID IS NOT NULL THEN 1 ELSE 0 END) AS ReadTraceOnly
FROM      [sqlnexus_managed].ReadTrace.tblUniqueStatements   m
FULL JOIN [sqlnexus_readtrace].ReadTrace.tblUniqueStatements r ON m.HashID = r.HashID;

-- 8. Connection counts by type (real logins vs CONNECTED BEFORE TRACE placeholders)
SELECT 'Managed'  AS Source,
       SUM(CASE WHEN ApplicationName = 'CONNECTED BEFORE TRACE' THEN 1 ELSE 0 END) AS FakeConns,
       SUM(CASE WHEN ApplicationName <> 'CONNECTED BEFORE TRACE' THEN 1 ELSE 0 END) AS RealConns
FROM [sqlnexus_managed].ReadTrace.tblConnections
UNION ALL
SELECT 'ReadTrace',
       SUM(CASE WHEN ApplicationName = 'CONNECTED BEFORE TRACE' THEN 1 ELSE 0 END),
       SUM(CASE WHEN ApplicationName <> 'CONNECTED BEFORE TRACE' THEN 1 ELSE 0 END)
FROM [sqlnexus_readtrace].ReadTrace.tblConnections;

-- 9. Batches with NULL BatchSeq→UniqueStore Seq join (validates spReporter_ExampleBatchDetails fix)
-- Should return 0 rows in both databases
SELECT 'Managed' AS Source, COUNT(*) AS OrphanedUniqueBatches
FROM [sqlnexus_managed].ReadTrace.tblUniqueBatches ub
WHERE NOT EXISTS (
    SELECT 1 FROM [sqlnexus_managed].ReadTrace.tblBatches b
    WHERE b.BatchSeq = ub.Seq
)
UNION ALL
SELECT 'ReadTrace', COUNT(*)
FROM [sqlnexus_readtrace].ReadTrace.tblUniqueBatches ub
WHERE NOT EXISTS (
    SELECT 1 FROM [sqlnexus_readtrace].ReadTrace.tblBatches b
    WHERE b.BatchSeq = ub.Seq
);

-- 10. Batch-to-connection linkage: batches missing a ConnSeq reference
-- Should be 0 or matching counts in both
SELECT 'Managed' AS Source, COUNT(*) AS UnlinkedBatches
FROM [sqlnexus_managed].ReadTrace.tblBatches b
WHERE b.ConnSeq IS NOT NULL
  AND NOT EXISTS (
      SELECT 1 FROM [sqlnexus_managed].ReadTrace.tblConnections c
      WHERE c.ConnSeq = b.ConnSeq
  )
UNION ALL
SELECT 'ReadTrace', COUNT(*)
FROM [sqlnexus_readtrace].ReadTrace.tblBatches b
WHERE b.ConnSeq IS NOT NULL
  AND NOT EXISTS (
      SELECT 1 FROM [sqlnexus_readtrace].ReadTrace.tblConnections c
      WHERE c.ConnSeq = b.ConnSeq
  );

-- 11. Statement-to-batch linkage: statements whose BatchSeq has no parent in tblBatches
SELECT 'Managed' AS Source, COUNT(*) AS OrphanedStatements
FROM [sqlnexus_managed].ReadTrace.tblStatements s
WHERE s.BatchSeq IS NOT NULL
  AND NOT EXISTS (
      SELECT 1 FROM [sqlnexus_managed].ReadTrace.tblBatches b
      WHERE b.BatchSeq = s.BatchSeq
  )
UNION ALL
SELECT 'ReadTrace', COUNT(*)
FROM [sqlnexus_readtrace].ReadTrace.tblStatements s
WHERE s.BatchSeq IS NOT NULL
  AND NOT EXISTS (
      SELECT 1 FROM [sqlnexus_readtrace].ReadTrace.tblBatches b
      WHERE b.BatchSeq = s.BatchSeq
  );

-- 12. Time range coverage: first/last event time in each database should be close
SELECT 'Managed'   AS Source, MIN(StartTime) AS FirstEvent, MAX(EndTime) AS LastEvent
FROM [sqlnexus_managed].ReadTrace.tblBatches
UNION ALL
SELECT 'ReadTrace', MIN(StartTime), MAX(EndTime)
FROM [sqlnexus_readtrace].ReadTrace.tblBatches;

Expected Differences

Some minor differences are acceptable:

• Hash IDs: Will differ (different hashing algorithms) — compare by NormText instead
• Normalization: Minor differences in how literals are parameterized (e.g., {STR} vs N'{STR}')
• Event counts: Should be within ~1-2% — differences may arise from how Starting events without matching Completed events are handled
• Aggregation boundaries: Time intervals may align slightly differently depending on the first event timestamp
• Connection tracking: ReadTrace.exe may handle MARS sessions differently

Red Flags (investigate if seen)

• Total batch count differs by more than 5%
• Top-10 queries by CPU have significantly different rankings
• Missing entire categories of events (e.g., zero statements, zero connections)
• Reports that work with ReadTrace but produce errors with TraceEventImporter

7. File Exclusion

• Place log_1.trc, log_2.trc, and *_blk.trc files alongside valid trace files
• Verify these are excluded from import (logged as excluded)
• Verify system_health, AlwaysOn_health XEL files are NOT picked up (only *pssdiag* and *LogScout* patterns)

8. Re-import & Drop Tables

• Import data, then import again with "Drop existing ReadTrace tables" checked
• Verify no duplicate data, schema is cleanly recreated

9. Disabled Importer

• Uncheck Enabled, re-import
• Verify trace files do not appear in file list and ReadTrace schema is not created

10. Cancellation

• Start an import of a large trace file set
• Click Stop during import
• Verify the import stops cleanly without leaving partial/corrupted data

[github-advanced-security]

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.