Quality Score

Abstract: The phenomenon of sponsored search advertising---where advertisers pay a fee to Internet search engines to be displayed alongside organic (nonsponsored) Web search results---is gaining ground as the largest source of revenues for search engines. Using a unique six-month panel data set of several hundred keywords collected from a large nationwide retailer that advertises on Google, we empirically model the relationship between different sponsored search metrics such as click-through rates, conversion rates, cost per click, and ranking of advertisements. Our paper proposes a novel framework to better understand the factors that drive differences in these metrics. We use a hierarchical Bayesian modeling framework and estimate the model using Markov Chain Monte Carlo methods. Using a simultaneous equations model, we quantify the relationship between various keyword characteristics, position of the advertisement, and the landing page quality score on consumer search and purchase behavior as well as on advertiser's cost per click and the search engine's ranking decision. Specifically, we find that the monetary value of a click is not uniform across all positions because conversion rates are highest at the top and decrease with rank as one goes down the search engine results page. Though search engines take into account the current period's bid as well as prior click-through rates before deciding the final rank of an advertisement in the current period, the current bid has a larger effect than prior click-through rates. We also find that an increase in landing page quality scores is associated with an increase in conversion rates and a decrease in advertiser's cost per click. Furthermore, our analysis shows that keywords that have more prominent positions on the search engine results page, and thus experience higher click-through or conversion rates, are not necessarily the most profitable ones---profits are often higher at the middle positions than at the top or the bottom ones. Besides providing managerial insights into search engine advertising, these results shed light on some key assumptions made in the theoretical modeling literature in sponsored search.

Abstract: The phenomenon of sponsored search advertising – where advertisers pay a fee to Internet search engines to be displayed alongside organic (non-sponsored) web search results – is gaining ground as the largest source of revenues for search engines. Using a unique 6 month panel dataset of several hundred keywords collected from a large nationwide retailer that advertises on Google, we empirically model the relationship between different sponsored search metrics such as click-through rates, conversion rates, cost-per-click, and ranking of advertisements. Our paper proposes a novel framework and data to better understand the factors that drive differences in these metrics. We use a Hierarchical Bayesian modeling framework and estimate the model using Markov Chain Monte Carlo (MCMC) methods. Using a simultaneous equations model, we quantify the relationship between various keyword characteristics, position of the advertisement and the landing page quality score on consumer search and purchase behavior as well as on advertiser’s cost-per-click and the search engine’s ranking decision. Specifically, we find that (i) retailer-specific keywords are associated with an increase in click-through and conversion rates while brand-specific keywords are associated with a decrease in click-through and conversion rates, (ii) the monetary value of a click is not uniform across all positions because conversion rates are highest at the top and decrease with rank on as one goes down the search engine results page, (iii) while search engines take into account the current period’s bid as well as prior click-through rates before deciding the final rank of an advertisement in the current period, the current bid has a larger effect than prior click-through rates, (iv) an increase in landing page quality scores is associated with an increase in conversion rates and a decrease in advertiser’s cost-per-click and (v) keywords that have more prominent positions on the search engine results page, and thus experience higher click-through or conversion rates are not necessarily the most profitable ones – profits are often higher at the middle positions than at the top or the bottom ones. Besides providing managerial insights into search engine advertising, these results shed light on some key assumptions made in the theoretical modeling literature in sponsored search.

Abstract: Objectives To describe the development, validation and inter-rater reliability of an instrument to measure the quality of patient decision support technologies (decision aids). Design Scale development study, involving construct, item and scale development, validation and reliability testing. Setting There has been increasing use of decision support technologies – adjuncts to the discussions clinicians have with patients about difficult decisions. A global interest in developing these interventions exists among both for-profit and not-for-profit organisations. It is therefore essential to have internationally accepted standards to assess the quality of their development, process, content, potential bias and method of field testing and evaluation. Methods Scale development study, involving construct, item and scale development, validation and reliability testing. Participants Twenty-five researcher-members of the International Patient Decision Aid Standards Collaboration worked together to develop the instrument (IPDASi). In the fourth Stage (reliability study), eight raters assessed thirty randomly selected decision support technologies. Results IPDASi measures quality in 10 dimensions, using 47 items, and provides an overall quality score (scaled from 0 to 100) for each intervention. Overall IPDASi scores ranged from 33 to 82 across the decision support technologies sampled (n = 30), enabling discrimination. The inter-rater intraclass correlation for the overall quality score was 0.80. Correlations of dimension scores with the overall score were all positive (0.31 to 0.68). Cronbach's alpha values for the 8 raters ranged from 0.72 to 0.93. Cronbach's alphas based on the dimension means ranged from 0.50 to 0.81, indicating that the dimensions, although well correlated, measure different aspects of decision support technology quality. A short version (19 items) was also developed that had very similar mean scores to IPDASi and high correlation between short score and overall score 0.87 (CI 0.79 to 0.92). Conclusions This work demonstrates that IPDASi has the ability to assess the quality of decision support technologies. The existing IPDASi provides an assessment of the quality of a DST's components and will be used as a tool to provide formative advice to DSTs developers and summative assessments for those who want to compare their tools against an existing benchmark.