SEOMoz' Long List of Link Searches --- The FirstStop Way

This post will demonstrate some of FirstStop WebSearch’s advanced features in a real world example based on a blog post Long List of Link Searches from SEOMoz where Rand Fishkin writes about the competitive intelligence aspects in SEO (Search Engine Optimization), particularly in link building.

As I read this article, I realized that FirstStop WebSearch fits well with the techniques he describes, so I decided to test-drive them.

Rand picked a website (EvoGear.com) and one of their primary keyword phrases "snowboard equipment". Then he illustrated how to perform searches that will result in high quality link acquisition targets.

His #1 rule is:

You should perform all of these searches at each of the major engines (Google, Yahoo!, MSN & Ask - yes, even Ask, as they often link to some very valuable and achievable link sources).

Good. In FirstStop I can search these four engines simultaneously (actually I can search many more engines all at once, but let's stick to these four in our test-drive).

Next step:

Basically, we're seeking every possible permutation of the term/phrase - links from any of the top ranking sites (#1-100, depending on competitiveness) will provide high value. As you go down the list, you can also use these modified terms/phrases to get extra results from the other sources.

snowboard equipment
"snowboard equipment"
snow board equipment
"snow board equipment"
snowboard
"snow board"
snowboards
"snow boards"

OK. This seems to be a good task for the FirstStop WebSearch batch search feature.

Since the four search engines have similar phrase syntax, we can use the Simple Batch wizard (menu Batch/Simple Batch).

Simple Batch dialog box

Just copy all the search terms into the text box. Change the search limits to 100 results per search engine (as Rand suggests). You can leave the default 500 results per search intact since any value more than 400 will work for four engines with 100 results maximum. Now click "OK".

Batch Editor dialog box

You will see eight searches prepared in the Batch Editor. Although it is not necessary, you might want to change the directory where the search results will be saved.

Now I'm ready to start the search. Just click the "Start" button.

FirstStop WebSearch will automatically perform all searches and compile the search results. Once the batch search is complete, you will see a report. Here is my report:

1. "snow board equipment" 292 results
    
2. "snow board" 353 results
    
3. "snow boards" 335 results
4. "snowboard equipment" 312 results
    
5. snow board equipment 376 results
    
6. snowboard 324 results
    
7. snowboard equipment 306 results
    
8. snowboards 272 results

FirstStop automatically removes duplicate links (site.com and www.site.com, and www.site.com/index.html are not considered as duplicates though), so the numbers in the report indicate only unique links. Now you can see why you should use more than one search engine; for the selected engines, about 75% of top 100 search results are unique.

Now let's get back to our search results and see what we can do with them using FirstStop WebSearch.

First of all, you can see all the recent searches saved in the "History" pane.

Search history pane

Double-click any of them to see the corresponding search results. You can click on the column headers to sort the results. For example, let's sort by "Hits" to see how many of the search results overlap. Results with "4" in the "Hits" column are full duplicates (they exist in top 100 search results of each search engine). Here are the numbers of full duplicates:

• snowboards - 13
    
• snowboard equipment - 3
    
• snowboard - 2
    
• snow board equipment - 0
    
• "snowboard equipment" - 5
    
• "snow boards" - 2
    
• "snow board" - 3
    
• "snow board equipment" - 9

Surprising, isn't it?

In the "Engines" column, you will find other SEO-specific information - the names of the search engine(s) where a link was found. In parenthesis, next to each search engine name, you will see the position of the link in the corresponding engine's search results.

Let's go further. On the left side of FirstStop you will see the Discovery Tree. It's a really useful tool for our task.

Discovery Tree with different branches expanded

Let's start with the "Search Sources" section. Here you can see the engine names and the number of search results FirstStop has retrieved from them. You can click an engine name and, in the search results list on the right, you will see only search results originating from that engine. Click "All Search Sources" to see all search results.

In the "Domains" section you will see the search results broken down by TLD (top level domains) and corresponding countries. You can expand the top level domains to see the web sites in those domains and the number of links to those web sites. Just click any site or TLD to restrict the search results to the selected site or domain.

The remaining section of the Discovery Tree breaks down the search results by words and phrases most frequently found in the search results. An SEO specialist can use this section to identify new variations of search terms and have a snapshot of keywords used on the competitive websites (we are talking about the competitive intelligence aspects of SEO, aren't we?).

The searches I just performed, Rand calls "obvious". Then he suggests that we proceed with the engines' advanced query parameters, such as:

allintitle:snowboard equipment
allinanchor:snowboard equipment
allinurl:snowboard equipment
allintext:snowboard equipment

You can use those parameters in the "Simple Search" mode, because, in this mode, FirstStop passes the search strings to search engines intact.

Search results for allinurl:snowboard equipment

To use the engines' advanced parameters in the Batch Search, make sure to select one search engine at a time, because each engine has its own set of advanced parameters.

Then Rand suggests using Alternative Search Sources:

The main indices are great ways to define value, but adding in some alternate sources for link searches can help to diversify. The engines might not always consider these sources as important (which is why I'd stick to only the highest profile sites/pages in these results), but they can often be great sources for traffic.

OK. As you know, FirstStop has many pre-installed search sources and almost any other search source can be added into FirstStop WebSearch.

The rest of Rand’s suggested searches (Directory Search Terms, Blog & Forum Searches, Submit-Type Searches) are just variations of the batch searches. Just compile your list of search terms and copy them into the "Simple Batch" dialog box.

As you can see, FirstStop fits well with the techniques Rand describes in his blog post. But why use FirstStop WebSearch instead of your web browser? There are quite a few good reasons to do this.

• FirstStop provides a single interface for searching multiple search sources.
• FirstStop automatically saves all searches.
    
• FirstStop has handy filtering and sorting tools.
    
• But the main reason is FirstStop saves a lot of time.

Just estimate the volume of searches. Even without the searches from the "Alternative Search Sources" section, there are 72 suggested Google searches in the blog post. That means 288 searches on all four major search engines.

If you don't know how much time that could take, use our online productivity calculator. For this task, enter the following values: 28800 results (288 searches, 100 results per search) and 100 results per page (as Rand suggested). The calculator estimates more than 12 hours of search if you do it in your web browser. You can even reduce the "Results Store Time" in the advanced option to only five seconds (you still need to save the search results, don't you?), but the task still takes more than seven hours to complete.

Seven hours of hard repetitive work before you can even start to work with the search results. On the other hand, you can have FirstStop do all this work for you in much less time.

Feel free to ask questions and request similar demonstrations of other FirstStop features. You can leave your comments here or use a contact form on our site.

Online productivity calculator: More visual and accurate results.

Last month we introduced the beta version of our online web search productivity calculator. We now have an updated beta version on the website. The most visible change is the productivity graph. Now you will not only see the time and money expended data based on your request, but also see how the productivity changes over the course of time. The less visible change is in the algorithm used in the calculator. Now it more accurately estimates the productivity of the manual approach. In our recent post Automated vs Manual. How much faster?, where we explained the mathematical computations used in our calculator, we introduced a fatigue coefficient that denotes how much slower a human searcher will process each successive page of search results. This coefficient takes into account the influence of fatigue and the likelihood that the searcher will get sidetracked or be disturbed in the middle of the task. In our productivity calculator, this coefficient is 3%. However, the original formula didn't reflect the fact that a searcher typically takes short breaks and goes to lunch. Let's assume that our fatigue coefficient covers the time of those breaks (it just distributes them evenly throughout the work time). But another aspect of "human nature" is, after getting some rest, we are usually fresh and full of energy. And our productivity is as high as in the beginning of the work day. The new algorithm takes into account the effect that these breaks have on an individual’s work output. When the time spans between successive pages of search results reach 10 minutes, the calculator assumes that the searcher has had some rest and her productivity is high again. You can see this effect on the graph if you enter, for example, 2000 search results and 10 results per page. Now test our online calculator and let us know what you think about it. We are also interested in real life stories about web search productivity. Did you try to manually compile large lists of search results? Or did you use FirstStop WebSearch Business Edition withmaximum "results per search engine/search" settings? How did your productivity compare with the results of our productivity calculator? You can either comment in this blog post or use a contact form on the online productivity calculator web page.

Automated vs Manual. How much faster?

Everyone knows that if the same task can be executed both by a man and a computer, the task will be completed much faster by a computer. The question is how much faster. The answer depends on many things, so let's be specific and estimate the difference for a task that is familiar to us – web search. If you don't want to read all the mathematical details, you can go straight to the conclusions, view the performance graphs (click the thumbnails), or check our online productivity calculator.

Productivity estimates graphs

If you want to know the details of our calculations, keep on reading. The task is to compile a list of search results. Here is the manual approach:

1. Open a search engine's web page, enter a query, and click “Search”. The timing begins here.
2. The web page with search results is being loaded. On slow connections and with slow search engines this may take some time. Let's call it “load time” (t_load).
3. Then the searcher should store the results somewhere – Word, Excel, or anything you use to store the gathered search results. This may take anywhere from a dozen seconds to several minutes (e.g., if you want to store URLs, Titles, and Descriptions of search results in separate columns of your spreadsheet or database). Let's call it “store time” (t_store).
4. Now the searcher clicks “Next” and repeats steps 2 and 3 for the next page of search results.
5. Lather, rinse, and repeat until the desired number of search results is gathered.

Here is the most simple and straightforward automated approach:

1. Open a program, enter a query, and click “Search”. The timing begins here.
2. As in the manual approach, the web page with search results will be compiled by the search engine and then downloaded by our program. This will most likely result in the same “load time” (t_load).
3. Then the program will to store the search results in its list. This will be the “automated store time” (t_{automated_store}).
4. Now the program loads the next page of search results.
5. Lather, rinse, repeat until the desired number of search results is gathered.
6. If needed, the complete list of search results can be exported to an application of your choice (e.g., Word, Excel, etc.).

Let's assume that the search engine provides N results per page. So the number of pages we need to process will be: N_pages = N_{total_results}/N_{results_per_page} So far the calculations seem to be pretty easy. The time needed to compile the list of search results will be: T_manual = N_pages * (t_load + t_store) and T_automated = N_pages * (t_load + t_{automated_store}) correspondingly. However, while the formula for the automated approach is rather accurate, it is not all that simple with humans. With such repetitive operations, we humans can quickly get tired and bored. It is easy for us to get sidetracked in the middle of a task. We can be interrupted by a phone call or a colleague. We might want to check some search result right away or go get some coffee. All this things make the actual total time for the manual approach greater than result you would obtain using the formula shown above. Let's add this notorious “human factor” to our formula. It is clear that the more pages the searcher needs to process, the more tired she gets, the more chances she will have to get sidetracked or be disturbed. So let's introduce a fatigue coefficient (k) that indicates how much slower the store time will be for each successive page. The fatigue coefficient of 1.05 indicates that it will take 5% more time to store the results for each successive page: e.g., first page - 20 seconds; second -21 seconds; third – 22.05, etc. Now the formula is more accurate but more complex at the same time. T_manual = n*t_load + t_store + t_store*k + t_store*k² + ... + t_store*k^n-1, where n is N_pages Most likely you will need a spreadsheet to calculate the total time for a significant number of pages. Or you can use our online productivity calculator. This calculator estimates the total times and RPM (results per minute) for both approaches and compares them in terms of saved time and money (check advanced options). The fatigue coefficient used in the calculator is 1.03 (3%). 8 hour work days used in the estimation for long lasting tasks. Conclusions

1. The productivity of an automated search solution doesn't depend on the number of search results you need to gather. The cost of processing each page is the same.
2. The productivity of human searchers decreases with every additional page of search results.

Which approach to choose?

• If you only need one page of search results, you shouldn't bother with any specialized solution. Just go to your favorite search engine.
• If you need to collect search results from about a dozen web pages, the advantages of an automated solution will be minimal. You should consider automation only if you often perform such tasks.
• If you need to process really large volumes of search results, an automated solution is the only reasonable choice. It would take a whole work day for a human search to collect results from 300 web pages, while a computer program can complete such a task in just half an hour.

Any questions and comments are welcome.

digg it : add to del.icio.us