It should not be confused with a vulnerability scan. In its Ransomware playbook, the Canadian Centre for Cyber Security distinguishes "vulnerability scanning to identify known vulnerabilities within applications" from "penetration testing, which simulates attacks that cybercriminals might perform to evaluate how well the infrastructure withstands them". A scan produces a list; a penetration test checks what an attacker can do with it.
Short answer
You describe the systems to test and why: your request is reviewed by the Courtier TI team, then entrusted to the IT services company that will prepare your quote. The scope, rules of engagement and written authorization are set with you before any testing.
What a penetration testing mandate can cover
- An external test of what is exposed to the Internet: website, remote access, email, public IP addresses
- An internal test starting from an already compromised workstation or account, to see how far an attacker could get
- Testing a web application or customer portal, with or without test accounts
- Testing Wi-Fi and the separation between the guest network and the business network
- A social engineering component, such as a targeted fake email, when agreed in writing
- The report, a walkthrough of the findings and a retest after remediation
How a penetration test works
- Scoping: the targets, exclusions and goal of the test are defined, for example validating remote access or a new application.
- Rules of engagement and written authorization: who authorizes the test, testing windows, allowed and prohibited techniques, contacts, and what to do if the tester discovers an incident in progress.
- Reconnaissance and analysis: the tester maps what is exposed and identifies potential vulnerabilities, often with scanning tools.
- Controlled exploitation: the tester tries to exploit those vulnerabilities within the agreed limits to show the real impact, such as access to data or administrator rights.
- Reporting: a summary for management, then the detail of each finding with its risk level, evidence and a remediation recommendation.
- Remediation and retest: your team or provider fixes the issues, then the tester confirms the weaknesses are closed.
In its ITSP.10.033 control catalogue, the Canadian Centre for Cyber Security states that all parties agree to the rules of engagement before penetration testing scenarios begin.
Questions to ask yourself before requesting a quote
- Why test now: a client, insurer or contract requirement, an application launch, a migration, or a need to validate your defences?
- Which systems are critical to the business: the online store, remote access, the file server, cloud accounts?
- Should the test start from outside, like an attacker on the Internet, or from inside, like an employee whose workstation was compromised?
- Will the tester receive information such as test accounts or diagrams, or start with no prior knowledge?
- Do you own all the target systems, or are some hosted by a third party whose agreement will be needed?
- Are the basics in place: automatic updates, two-factor authentication, backups? If not, the test will mostly confirm what you already know.
Risks and pitfalls specific to penetration testing
- A scan sold as a penetration test. The Centre notes that penetration testing goes beyond automated vulnerability scanning. Ask how much of the work is manual and whether weaknesses will be exploited or only listed.
- Testing without written authorization. The Criminal Code (s. 342.1) makes unauthorized use of a computer an offence. Authorization must come from someone with authority over each target system, including at your hosting providers.
- A vague scope. A test limited to the website says nothing about remote access or cloud accounts, and a poorly defined scope leads to disputed findings.
- Disrupting operations. Some techniques can slow down or interrupt a service. Plan testing windows, a recent backup and a reachable contact during the test.
- Exposing data. According to the Centre, penetration testing may expose information protected by laws or regulations to the testers, and rules of engagement or contracts can set out how to protect it. If personal information is communicated to the tester, Québec's section 18.3 requires a written contract that specifies the protection measures.
- A tester marking their own work. The Centre describes impartial testers, with no conflicts of interest with respect to the development, operation or management of the systems tested. Avoid giving the test to the team that runs those systems.
- A report left in a drawer. Without someone responsible for remediation and a retest, the weaknesses stay open.
What to specify in your request
- The targets: web addresses, exposed IP addresses, applications, sites and networks in scope
- The type of test: external, internal, web application, Wi-Fi, social engineering, or to be determined with you
- The information given to the tester and the test accounts available
- Constraints: hours to avoid, fragile systems, whether a test environment exists
- Hosting providers or vendors whose agreement is needed
- External requirements to meet and the expected report format
- Whether a retest after remediation should be included
- Who will sign the authorization and who to contact during the test
Frequently asked questions
What is the difference between a penetration test and a vulnerability scan?
A scan identifies known vulnerabilities, usually with automated tools. A penetration test tries to exploit them, as an attacker would. In its ITSP.10.033 catalogue, the Canadian Centre for Cyber Security says penetration testing goes beyond automated vulnerability scanning and is conducted by people with demonstrable skills and experience in network, operating system or application-level security.
How often should we run a penetration test?
There is no single frequency. The ITSP.10.033 catalogue leaves it to the organization to define the frequency and target systems, and notes that testing is especially important when moving from older technologies to newer ones. We suggest planning one after a major change, such as a new Internet-facing application, a cloud migration or a network overhaul, and whenever a client or contract requires it.
What should a good penetration test report contain?
A plain-language summary for management; the scope and methods actually used; each finding with its risk level, evidence and steps to reproduce it; prioritized remediation recommendations; and, after the retest, the status of each weakness. Ask for an anonymized sample report before accepting the quote.
Can the test disrupt our operations?
It can, especially for an internal test or older equipment. The rules of engagement exist to manage this: testing windows, excluded techniques, fragile systems flagged, a reachable contact and a stop procedure. A recent, tested backup before the start is a good precaution.
Does a penetration test make us compliant with Quebec's Law 25?
No. Section 10 requires reasonable security measures given, among other things, the sensitivity of the information, without prescribing penetration testing. A test helps verify your measures, but it replaces neither the incident register nor the privacy impact assessment. Our guide Law 25: what Quebec SMBs must do on the IT side covers the obligations.