How to research search keywords and competing pages

Choose the page you want to improve before buying keyword data. A comparison page for a scheduling tool needs different evidence from a help article about calendar permissions. Without that decision, an agent can return a large keyword spreadsheet that gives the writer little direction.

This workflow produces a short brief: a search question, the pages currently answering it, gaps you can substantiate, and a proposed page structure. The examples below describe a workflow; they are not a completed ranking study.

Give the agent a page and a market

Connect through Vaaya's installation guide, then paste this with your own details:

Research search demand for [page or product] in [country and language].
Audience: [specific buyer or user]. Start with these five questions: [list].
Inspect the search results and read relevant competing pages.
Return one brief with source URLs, observation dates and unresolved facts.
Separate provider estimates from observed search positions.
Use a total research budget of [amount], with a ceiling on every call.
Do not publish pages or change my website.

Define what a usable brief contains before expanding the query list. For a small first pass, five questions and a few relevant pages are enough to reveal whether you are researching instructions, comparisons, products or local services.

Follow the evidence into the page

1. Capture a search snapshot. Ask consult for serper/search using the exact question. The current schema supports q, gl, hl, location, num and page. Keep the country and language fixed. Record a device setting only when the selected route explicitly supports and honors it; otherwise mark device as unspecified.

2. Separate result types. Record organic links independently from ads, local results and other search features. Preserve the position supplied by the response rather than treating every displayed item as the same ranking list. Search snippets help choose pages to read, but cannot establish what the full article contains.

3. Read the relevant pages. Use OneScrape for a small set of known URLs, or ask consult for a structured extractor. Save the URL, retrieval time, page title and useful passages. Identify the job each page serves: a definition, an implementation guide, a product comparison or a purchase page.

4. Add keyword data selectively. If volume or related queries would change the decision, ask for an appropriate spyfu/query operation. Its path and operation-specific parameters must come from the current catalog. SpyFu documents domain statistics with a country setting and describes traffic figures as estimates. Keep that source label beside the number.

5. Write a page brief from the evidence. State what the reader should finish knowing or doing. Recommend sections that answer observed questions and specify the original examples, screenshots or product facts you still need. A competing heading alone is insufficient reason to copy its structure.

Keep one row per search question

Use this as an output contract for the research table:

query | country | language | location | device_or_unspecified
observed_at | organic_position | source_url | page_type
question_answered | missing_evidence | volume_estimate | estimate_source
proposed_page | editorial_reason | next_action

Put unknown values in the table explicitly. If a page cannot be read, record the blocked URL and limit the observation to its search snippet. A source that failed to load should never acquire a confident summary.

A useful editorial reason is specific: “The results explain setup, but our product requires an extra permission step that we can demonstrate.” “Competitors have long articles” does not explain what you should write.

Stop when the next query cannot change the brief

Count search, extraction and optional keyword calls against one research allowance. Cache page reads by URL and retrieval time so several queries can reuse the same evidence. An expensive retry needs a stated question it could resolve.

If results look unrelated, first check the exact query and location. If keyword data uses another market or month, retain it separately. Re-running unchanged calls until a preferred ranking appears makes the brief harder to trust.

Give the writer the proposed outline, source table and missing evidence list together. After publication, assess the page using your own search-performance data over a defined period. Keep this initial snapshot so later changes can be compared against the research that informed the page.

Questions

Does a high search volume make a keyword worth targeting?

No. Check whether the searcher needs what your page offers, whether you can answer the question, and what competing pages already provide. Volume is one provider estimate.

Does this workflow guarantee better rankings?

No. It produces an inspectable research brief. Search results change, and publishing a page does not guarantee a ranking or traffic outcome.

Can I compare results from different countries?

Yes, but keep each country, language, location and observation time separate. Do not combine unlike searches into a single rank.