Online productivity calculator: More visual and accurate results.

Last month we introduced the beta version of our online web search productivity calculator. We now have an updated beta version on the website. The most visible change is the productivity graph. Now you will not only see the time and money expended data based on your request, but also see how the productivity changes over the course of time. The less visible change is in the algorithm used in the calculator. Now it more accurately estimates the productivity of the manual approach. In our recent post Automated vs Manual. How much faster?, where we explained the mathematical computations used in our calculator, we introduced a fatigue coefficient that denotes how much slower a human searcher will process each successive page of search results. This coefficient takes into account the influence of fatigue and the likelihood that the searcher will get sidetracked or be disturbed in the middle of the task. In our productivity calculator, this coefficient is 3%. However, the original formula didn't reflect the fact that a searcher typically takes short breaks and goes to lunch. Let's assume that our fatigue coefficient covers the time of those breaks (it just distributes them evenly throughout the work time). But another aspect of "human nature" is, after getting some rest, we are usually fresh and full of energy. And our productivity is as high as in the beginning of the work day. The new algorithm takes into account the effect that these breaks have on an individual’s work output. When the time spans between successive pages of search results reach 10 minutes, the calculator assumes that the searcher has had some rest and her productivity is high again. You can see this effect on the graph if you enter, for example, 2000 search results and 10 results per page. Now test our online calculator and let us know what you think about it. We are also interested in real life stories about web search productivity. Did you try to manually compile large lists of search results? Or did you use FirstStop WebSearch Business Edition withmaximum "results per search engine/search" settings? How did your productivity compare with the results of our productivity calculator? You can either comment in this blog post or use a contact form on the online productivity calculator web page.

Automated vs Manual. How much faster?

Everyone knows that if the same task can be executed both by a man and a computer, the task will be completed much faster by a computer. The question is how much faster. The answer depends on many things, so let's be specific and estimate the difference for a task that is familiar to us – web search. If you don't want to read all the mathematical details, you can go straight to the conclusions, view the performance graphs (click the thumbnails), or check our online productivity calculator.

Productivity estimates graphs

If you want to know the details of our calculations, keep on reading. The task is to compile a list of search results. Here is the manual approach:

1. Open a search engine's web page, enter a query, and click “Search”. The timing begins here.
2. The web page with search results is being loaded. On slow connections and with slow search engines this may take some time. Let's call it “load time” (t_load).
3. Then the searcher should store the results somewhere – Word, Excel, or anything you use to store the gathered search results. This may take anywhere from a dozen seconds to several minutes (e.g., if you want to store URLs, Titles, and Descriptions of search results in separate columns of your spreadsheet or database). Let's call it “store time” (t_store).
4. Now the searcher clicks “Next” and repeats steps 2 and 3 for the next page of search results.
5. Lather, rinse, and repeat until the desired number of search results is gathered.

Here is the most simple and straightforward automated approach:

1. Open a program, enter a query, and click “Search”. The timing begins here.
2. As in the manual approach, the web page with search results will be compiled by the search engine and then downloaded by our program. This will most likely result in the same “load time” (t_load).
3. Then the program will to store the search results in its list. This will be the “automated store time” (t_{automated_store}).
4. Now the program loads the next page of search results.
5. Lather, rinse, repeat until the desired number of search results is gathered.
6. If needed, the complete list of search results can be exported to an application of your choice (e.g., Word, Excel, etc.).

Let's assume that the search engine provides N results per page. So the number of pages we need to process will be: N_pages = N_{total_results}/N_{results_per_page} So far the calculations seem to be pretty easy. The time needed to compile the list of search results will be: T_manual = N_pages * (t_load + t_store) and T_automated = N_pages * (t_load + t_{automated_store}) correspondingly. However, while the formula for the automated approach is rather accurate, it is not all that simple with humans. With such repetitive operations, we humans can quickly get tired and bored. It is easy for us to get sidetracked in the middle of a task. We can be interrupted by a phone call or a colleague. We might want to check some search result right away or go get some coffee. All this things make the actual total time for the manual approach greater than result you would obtain using the formula shown above. Let's add this notorious “human factor” to our formula. It is clear that the more pages the searcher needs to process, the more tired she gets, the more chances she will have to get sidetracked or be disturbed. So let's introduce a fatigue coefficient (k) that indicates how much slower the store time will be for each successive page. The fatigue coefficient of 1.05 indicates that it will take 5% more time to store the results for each successive page: e.g., first page - 20 seconds; second -21 seconds; third – 22.05, etc. Now the formula is more accurate but more complex at the same time. T_manual = n*t_load + t_store + t_store*k + t_store*k² + ... + t_store*k^n-1, where n is N_pages Most likely you will need a spreadsheet to calculate the total time for a significant number of pages. Or you can use our online productivity calculator. This calculator estimates the total times and RPM (results per minute) for both approaches and compares them in terms of saved time and money (check advanced options). The fatigue coefficient used in the calculator is 1.03 (3%). 8 hour work days used in the estimation for long lasting tasks. Conclusions

1. The productivity of an automated search solution doesn't depend on the number of search results you need to gather. The cost of processing each page is the same.
2. The productivity of human searchers decreases with every additional page of search results.

Which approach to choose?

• If you only need one page of search results, you shouldn't bother with any specialized solution. Just go to your favorite search engine.
• If you need to collect search results from about a dozen web pages, the advantages of an automated solution will be minimal. You should consider automation only if you often perform such tasks.
• If you need to process really large volumes of search results, an automated solution is the only reasonable choice. It would take a whole work day for a human search to collect results from 300 web pages, while a computer program can complete such a task in just half an hour.

Any questions and comments are welcome.

digg it : add to del.icio.us

FirstStop WebSearch v5.0 Released.

The long awaited version 5 of FirstStop WebSearch has been released. The new version makes FirstStop more powerful and more focused on increasing web searchers' productivity in business environments. What's new in version 5.0?

• Improved search engine specification allows users to work with more search engine types and obtain an increased level of detail in search results.
• Improved support for international characters and non-English language searches.
• Batch searching. Multiple searches can be entered as a batch, and FirstStop will automatically retrieve and save search results for every search in the batch.
• RSS support. RSS search feeds can be easily added to FirstStop WebSearch. FirstStop WebSearch can export search results to RSS 2.0 format.
• New Discovery Tree enhancements help break down search results not only by keywords and phrases, but also by domains and countries, and search sources.
• Handy desktop search bar provides users with a quick way to start searches whenever you need it without having to launch FirstStop WebSearch.
• New Configuration Wizard that helps new users get started.
• Many more minor modifications and improvements.

You can download a free 30 day trial version here.