Reference
The competitor engine
How ScopeMCP finds keywords a competitor ranks for and you do not, without buying a keyword database.
Why build it rather than buy it
A paid keyword API is the single biggest recurring cost in this category, and it is the reason most tools in this space either charge per month or quietly show you numbers they cannot substantiate.
Two free sources cover most of it. Bing Webmaster Tools exposes a keyword statistics endpoint that returns real monthly search counts. Google Programmable Search returns the live result page for any query, which tells you who actually ranks today rather than who ranked when someone last built a database.
Combining them gives measured demand plus observed position, which is what a gap analysis actually needs. What it does not give you is a competitor backlink index, a difficulty score derived from a proprietary model, or estimated competitor traffic. The tools never claim to.
How candidates are built
- Your own queries — the best free demand signal you already own, pulled from Search Console.
- Bing related keywords — free expansion from any seed.
- Your seeds — anything you pass in explicitly.
- Lexical derivation — a cheap secondary pass over your observed queries that needs no quota at all.
How a gap is confirmed
A keyword is only returned as a confirmed gap when a tracked competitor appears in the checked depth of the live result page and you do not. If your own data shows you already rank somewhere for that query, it is downgraded to contested rather than reported as new territory.
Candidates whose result page could not be fetched come back as unverified, with a caveat saying exactly that. The response reports how many candidates were checked versus skipped, so a partial run is never mistaken for a complete one.
Quota reality
The free Programmable Search tier allows 100 result pages per UTC day. That is enough for scheduled monitors and a handful of on-demand runs. Each call reports its own cost and your remaining balance, so you can see a budget problem before you hit it.
Cached pages, monitors reading stored snapshots, and any keyword already fetched in the last six hours do not consume quota.