Check who earns money from the recommendation, whether the author says what was actually tested and how, which sources back each claim, and when the page was last checked. A review with no disclosure, no method and no dates tells you little, however confident it sounds.
Search for any CMS and you will find hundreds of comparisons, top-ten lists and reviews. Some are careful work. Some are written mainly to earn commission or to rank in search. You do not need to distrust everything, but you do need a way to sort the useful pages from the rest. This guide gives you five questions to ask of any review. They apply equally to ours, and we explain how we handle them on our how we test page.
Who benefits if you follow this recommendation?
Start with the money. A review site can earn from affiliate commissions, sponsorship, free accounts or direct ties to a vendor. None of that automatically makes the content wrong, but it affects what gets included, what is ranked first and what is left out.
The US Federal Trade Commission’s endorsement guidance says that if there is a connection between an endorser and the marketer that a significant minority of consumers would not expect, and that would affect how they evaluate the endorsement, the connection should be disclosed clearly and conspicuously. It gives paid relationships, free products received, affiliate commissions and employment relationships as examples of connections that may need disclosure.
On affiliate links, the same guidance says a reviewer should disclose their relationship to the retailer clearly and conspicuously on their site, and that “Paid link” right next to an affiliate link should be an adequate disclosure. It also says an effective disclosure should be simple, like “This is an ad for BRAND”, and hard to miss instead of buried in comments or descriptions.
That is US guidance, and rules differ elsewhere. As a reader, use it as a test:
- Is there a disclosure at all?
- Is it near the recommendation, or hidden in a footer?
- Does the page say whether the ranking order is influenced by payment?
- Do the “best” picks have affiliate programmes, and do the good free options appear without them?
Google’s own guidance asks that paid links carry the sponsored value of the rel attribute, which is a signal to search engines, so a page that does not mark its commercial links is worth a second look. It is not proof, because readers do not see the attribute. Our contributor submissions guide covers the publisher’s side of the same issue.
What exactly did the reviewer test?
“We tested” is the most overused phrase in software reviews. Ask what was done, on what and for how long.
A credible testing claim has these features:
- A named version. “WordPress” is not a test subject. A numbered release is.
- An environment. Hosted or self-hosted, which server, which plugins.
- A task list. What was attempted: install, publish, migrate, extend.
- Evidence. Dated screenshots, logs or a repeatable procedure.
- Failures. If nothing went wrong, a reader should wonder how closely it was examined.
If a review has none of these, the author is probably summarising vendor material. That is a legitimate thing to do, but it should say so. A page that describes features “according to the vendor’s documentation” is more honest than one that implies hands-on experience it does not have.
Where do the facts come from?
Every number, price and feature in a comparison came from somewhere. Good reviews show where.
Prefer primary sources: the vendor’s documentation, pricing page, changelog and licence text. Be careful with a third-hand chain, such as a blog citing a blog citing a press release. Popularity figures, usage percentages and market-share claims need particular care, because they come from different methods and change over time.
Watch for specific signs of weak sourcing:
- Statistics with no source or a source that does not say it.
- Quotes with no named speaker.
- Feature lists that match the vendor’s marketing word for word.
- “Fully supports” claims with no detail.
You can check a claim yourself in a few minutes. Open the cited page, find the sentence and see whether it says what the review says it does. Our licensing guide and cost guide show how we link each fact to a page we checked.
When was this written and checked?
CMS software changes constantly. Features move between paid and free tiers, licences change and add-ons are dropped. A comparison that was right two years ago may mislead you today.
Look for:
- A publication date and a last-checked date, not only one of them.
- The versions covered.
- Signs of upkeep, such as a changelog or a note describing what was updated.
A page with a recent “updated” date but old content is a common trick. Check one or two details yourself against the vendor’s current pages before you trust the rest.
How was the list built?
“Top 10” lists hide their method. Ask what the criteria were and whether the weights make sense for you. A list of best CMSs for small blogs does not help a newsroom with a dozen editors and an approval workflow.
Look for lists that name a use case, state their criteria and include honest weaknesses for each entry. Be wary of lists where every entry is “great” and the differences are in adjectives. Our feature checklist and comparison pages take the opposite approach, with the same questions asked of every product.
What should you do with a review once you have read it?
Use it to build a shortlist, not to make the decision. Then:
- Confirm key facts in the vendor’s documentation.
- Check the licence and the real cost over three years.
- Run a trial install with your own content and your own editors.
- Ask for references from publishers of your size, and see what real publishers say about their CMS for how we handle that kind of claim.
A review is one input. Your own test, run on your own work, will always be more relevant.
Questions people ask
Are affiliate reviews untrustworthy?
Not automatically. Many are careful and useful. But an affiliate link means the author is paid when you click, so check that the relationship is disclosed and that the review explains what it tested and what it did not recommend.
How can I tell whether a reviewer really tested the software?
Look for specifics only a user would know: versions, settings, screenshots with dates, problems met and how they were fixed. If a review says it tested something but names no version, environment or method, treat the claim with caution.
How old is too old for a CMS comparison?
CMS products change features, pricing and licensing between releases, so a comparison with no date, or one more than a year old, needs checking against current vendor pages. Ask which version it covers.
Should I trust ratings and star scores?
Treat them as a rough sign of popularity, not quality for your needs. Ask who gave the rating, how many people did, and whether the vendor invited or rewarded reviews.
Sources
- The US Federal Trade Commission says that if there is a connection between an endorser and the marketer that a significant minority of consumers would not expect and that would affect how they evaluate the endorsement, the connection should be disclosed clearly and conspicuously. — source, checked 10 October 2026.
- The FTC's endorsement guidance gives paid relationships, free products received, affiliate commissions and employment relationships as examples of connections that may need disclosure. — source, checked 10 October 2026.
- The FTC's endorsement guidance says a reviewer should disclose their relationship to the retailer clearly and conspicuously on their site, and that 'Paid link' right next to an affiliate link should be an adequate disclosure. — source, checked 10 October 2026.
- The FTC's endorsement guidance says an effective disclosure should be simple, like 'This is an ad for BRAND', and should be hard to miss rather than buried in comments or descriptions. — source, checked 10 October 2026.
- Google's guidance says to mark links that are advertisements or paid placements (commonly called paid links) with the sponsored value of the rel attribute. — source, checked 10 October 2026.