Site Testing is CanaryWP’s primary interface for verifying a WordPress environment.
What Site Testing can check
Depending on the selected tests, CanaryWP can evaluate:
- Site boot
- Fatal-error screens
- Public pages
- Navigation
- Assets
- Screenshots
- WP Admin
- Login
- Content
- Key page behavior
- Forms
- Safe functionality
- Visual differences
Before a test
Make sure:
- The correct instance is selected.
- The instance is running.
- The site can be reached.
- WP Admin credentials are available if the test requires them.
- The comparison URL is correct if visual regression will be used.
Baseline testing
Run a baseline before a significant change.
The baseline tells you which failures already existed.
Without it, a failed post-update test can be difficult to attribute.
Run tests
- Open Site Testing.
- Select the instance.
- Provide the source/original URL if required for comparison.
- Select the tests relevant to the change.
- Start the run.
- Watch the test and step statuses.
- Wait for completion before evaluating the final result.
Test states
Individual work can move through states such as:
- Pending
- Running
- Done
- Failed
A failed step should be investigated individually. It can also be manually passed if deemed minor failure.
Run summary
Use the summary to understand:
- Overall result
- Pass count
- Fail count
- Progress
- Start time
- Visual-diff information where available
Step details
Step details can include:
- Status
- Duration
- Last update
- Error message
- Screenshot
- Comparison screenshot
- Diff image
Visual regression
A visual difference means the rendered result changed.
It does not automatically mean the change is bad.
Review whether the difference is:
- Intended
- Harmless dynamic content
- Layout shift
- Missing content
- Styling regression
- Rendering failure
After an update
Run the tests most likely to be affected by the change.
For a major WordPress or PHP update, a wider regression run is appropriate.
Rerun when
Rerun tests after:
- Import
- Plugin change
- Theme change
- WordPress change
- PHP change
- Database operation
- Content change
- Cache rebuild
- Relevant DNS/configuration change
Failed tests
For a failed test:
- Read the step message.
- Open the affected page manually.
- Review screenshots.
- Review logs.
- Determine whether the failure existed in the baseline.
- Correct the underlying issue.
- Rerun the affected test.