Security · Topic 23 of 23 · Part IV — From findings to value

Bad vs. good reporting + drafting practice

The same structure can drive change in an engineering team — or sit unread in a SharePoint folder. The difference is craft.

Syllabus: § Reporting (6.4) → Bad and good reporting practices
Topic 23 · Report quality

Same structure. Opposite quality.

By the end of this topic you can:
  • recognize the dominant patterns of bad reporting and articulate why each fails the reader
  • recognize the patterns of good reporting and articulate why each works
  • critique a sample report and produce concrete revisions
  • draft a finding from raw observations to a client-ready form
  • apply tone, layout, and style guidelines that make reports usable across audiences

Bad reporting: copy-paste antipatterns

Scanner dump

Raw Nessus / Qualys output pasted with a cover page. No triage, no analysis, no environmental adjustment. False positives sit alongside real findings.

Why it fails: the client paid for analysis, not log forwarding.

CVE dictionary

Each finding is a copy of the NVD advisory. No context about how the CVE manifests in this environment; remediation is the generic CVE suggestion.

Why it fails: any reader could look up CVEs. The tester's value is the application to the specific environment.

OWASP-quoted report Each finding quotes the OWASP Top 10 page text with no specific request, endpoint, or application-specific reproduction. The OWASP Top 10 is a risk-awareness document — quoting it is not a finding.

Bad reporting: evidence and process failures

Pasted screenshot

Terminal or browser screenshots without captions, context, or surrounding analysis. The reader has to guess what command produced the output and what to conclude.

Evidence without framing is not evidence.

"We ran X" report

A list of tools and commands used during the engagement presented as findings: "Nmap was run. Burp Suite was used. Nessus completed in 4 hours."

Process documentation is not findings. The client cares what was discovered, not what was run.

Boilerplate remediation Every SQL injection finding says "use parameterized queries"; every XSS says "validate and encode all input." The engineer cannot act without further research — which the tester should have done once.

Bad reporting: severity and language failures

Everything is Critical

30 findings, 25 marked Critical. No environmental adjustment, no compensating controls considered.

When everything is Critical, nothing is. The client learns to discount the tester's severity ratings entirely.

Hedged into meaninglessness

"It may be possible that an attacker could potentially be able to exploit X, depending on conditions, which would result in a possibly significant impact."

The reader cannot extract a position. Hedging beyond what evidence requires is a failure of professional judgement.

Unfindable-action report Technically thorough but the reader cannot extract "what do I do?" — no prioritization, no executive summary, no top recommendations. Action is not optional.

Bad reporting: structural failures

  • Unjustifiable-claim report — findings claim more than the evidence supports. When a developer cannot reproduce the claim, the entire report's credibility collapses.
  • Audience-blind report — too technical for executives, too superficial for engineers. A flat document serves no one.
  • "Report = findings" report — no executive summary, no methodology, no scope statement, no attack narrative. Individual findings may be fine; collectively they don't form a document.
  • Padded-page-count report — long appendices of scanner output, glossaries of known terms, multi-page tool-version tables. Clients learn to skim; trust erodes.
  • "We found nothing" failure — no statement of what was tested, what controls held, what the absence means. An engagement with no findings can still demonstrate value through scope, methodology, and confident absence-of-findings framing.

Good reporting: analysis and evidence

Curated findings Triaged, contextualized, scored with environmental awareness, prioritized. False positives filtered. High signal-to-noise.
Environment-specific descriptions The vulnerability is described in this system, with this configuration. CVE/CWE is a reference, not the content.
Concrete evidence with frames Screenshots have captions. Requests and responses are quoted in full. The reader is shown what the tester saw and told what to conclude.
Reproducibility Findings can be reproduced from the report alone. Requests and pre-conditions are documented. Peer-reviewable.

Good reporting: remediation, severity, and action

Specific, framework-aware remediation "In the Django application, replace Cart.objects.raw(...) calls in cart/views.py (lines 47–53) with parameterized querysets. The same pattern appears in orders/views.py lines 134–139; review both."
  • Defensible severity — each Critical and High justifies its rating in the impact section; environmental adjustments made explicit.
  • Confident, direct claims — "The application accepts unauthenticated requests to the administrative endpoint." Not hedged versions of the same claim.
  • Action visible at every level — executive summary names top recommendations; each finding has remediation. The reader can stop at any depth and know what to do.
  • Audience routing — executive summary for executives; detailed findings for engineers; scope and methodology for auditors.

Tone calibration

AvoidUse instead
"The development team failed to implement basic security controls" "The application is missing input validation at the affected endpoints"
"We owned the domain in two hours" "Domain compromise was achieved within the engagement window via the path described in Section 4"
"A skilled attacker could leverage this to devastating effect" "If exploited, this vulnerability would allow read access to the customer-data table, containing approximately N records"
"We got root real quick" "Local privilege escalation to root was achieved in approximately 8 minutes"

Reports are read by the people who built the system. Frame findings about systems, not people.

Critique exercise: one finding, two versions

Bad version

Finding 7 — XSS | Severity: High

The application has cross-site scripting. According to OWASP, XSS is a category of vulnerability where untrusted input is rendered in users' browsers. We found XSS on the search page.

Remediation: Validate and encode all input.

References: OWASP Top 10 A03.

  • Title names technique, not impact
  • Generic OWASP boilerplate
  • No endpoint, parameter, payload, or evidence
  • Severity unjustified; remediation generic

Good version

F-007 — Reflected XSS in product-search endpoint enables session hijacking of authenticated users

Severity: High (CVSS 3.1: 7.4 — AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:L/A:N)
Affected: https://app.example.com/products/search?q=…

The q parameter is reflected without contextual output encoding. The permissive CSP ('unsafe-inline') allows inline script execution. The session cookie lacks HttpOnly; the XSS payload can therefore exfiltrate it.

Remediation: Remove |safe from templates/search.html line 23. Set SESSION_COOKIE_HTTPONLY = True in settings/production.py.

Self-critique checklist

  • Does the title convey impact, not just technique?
  • Is the affected asset named specifically?
  • Is the description specific to this environment?
  • Does the evidence support the claim, reproducibly?
  • Does the impact statement match the severity?
  • Is the remediation actionable — could the engineer open the file and make the change?
  • Are CVE/CWE references used as references, not content?
  • Is the tone factual and professional?
  • Would I defend each claim in front of an independent reviewer?
  • Would the developer who built this find this useful or insulting?
Drafting practice cycle Write from scratch → self-critique → revise → peer-review → revise again. Repeat across finding types: technical vulnerability, configuration issue, missing control, chained narrative.

Retest and peer review

The retest as quality test

  • A clean retest reproduces each finding from the report's reproduction steps, then observes whether the fix is in place.
  • If the retest cannot reproduce: reproduction steps were incomplete.
  • If the retest cannot find the asset: the report was not specific enough.
  • If the retest cannot tell whether the fix addresses root cause: the report described symptoms, not vulnerability.

Write with the retest in mind.

Peer review

Mature testing organizations require peer review before delivery. A second tester reads the report, checks evidence against claims, raises ambiguities, catches errors.

A junior tester who self-edits is producing a draft. Senior review on every report accelerates development. The habit, once built, persists into senior practice.

Cost vs. value Cost: hours. Value: consistently meeting a quality bar before the client sees the report.

Check — peer-review in order

Reflection

Walk through how you would peer-review a colleague's report. What do you check, in what order?

Reveal answer

(1) Executive summary — does it pass the 10-minute test? (2) Scope & methodology — accurate; limitations honest? (3) Per finding: title conveys impact? evidence supports claim? severity defensible? remediation actionable? reproduction complete? (4) Attack narrative — does it land? ATT&CK consistent? (5) Prioritisation — top recommendations actually doable? (6) Tone and style — moralizing, hedging, casual language. (7) Reproducibility test — pick one finding, mentally reproduce from the text alone.

What you take home

  • Bad reports share a small catalogue of antipatterns — learn to name them and you can diagnose any report fast.
  • Good reports are specific: specific findings, specific evidence, specific remediation for the specific environment.
  • Severity and claims must be defensible — write to the bar of an independent reviewer.
  • Tone is professional observation, not moralizing, celebrating, or marketing language.
  • Reproducibility is the ultimate quality test: can someone retest from the report alone?
  • Peer review is not optional in mature organizations — the habit compounds over a career.
  • "Would the developer who built this system find this useful?" is the most useful question to ask before submitting.

The report is the engagement. Make it worth reading.

END · TOPIC 23

Write so it drives change.

Apply the self-critique checklist to your next draft before anyone else sees it.