literature review on search engines

Search Engines in Learning Contexts: A Literature Review

• Jaures Styve Kameni Homte National Advanced School of Engineering, University of Yaounde I https://orcid.org/0000-0003-0367-0408
• Bernabé Batchakui National Advanced School of Engineering, University of Yaounde I
• Roger Nkambou Université du Québec à Montréal, Montreal

How to Cite

• Endnote/Zotero/Mendeley (RIS)

Copyright (c) 2021 KAMENI HOMTE JAURES STYVE

This work is licensed under a Creative Commons Attribution 4.0 International License .

The submitting author warrants that the submission is original and that she/he is the author of the submission together with the named co-authors; to the extend the submission incorporates text passages, figures, data or other material from the work of others, the submitting author has obtained any necessary permission. Articles in this journal are published under the Creative Commons Attribution Licence (CC-BY What does this mean? ). This is to get more legal certainty about what readers can do with published articles, and thus a wider dissemination and archiving, which in turn makes publishing with this journal more valuable for you, the authors. By submitting an article the author grants to this journal the non-exclusive right to publish it. The author retains the copyright and the publishing rights for his article without any restrictions.

Information

• For Readers
• For Authors
• For Librarians

#9 in Google Scholar's Top publications in Educational Technology

Other journals

Your article doesn't fit this journal's scope? Have a look at our other journals: https://online-journals.org

International Journal of Emerging Technologies in Learning (iJET) – eISSN: 1863-0383

Indexed in Ei Compendex, EBSCO, dblp, LearnTechLib. Long-term archiving is assured by Portico. Plagiarism check by iThenticate. Published under CC-BY.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Advanced Search
• Journal List
• Int J High Risk Behav Addict
• v.1(4); Winter 2013

Comparison of Four Search Engines and their efficacy With Emphasis on Literature Research in Addiction (Prevention and Treatment)

Gholam reza samadzadeh.

1 Department of Library and Medical Information Sciences, Zahedan University of Medical Sciences and Health Services, Zahedan, IR Iran

Tahereh Rigi

2 Department of Library and Information Sciences, Zahedan Payame Noor University, Zahedan, IR Iran

Ali Reza Ganjali

3 Research Center of Children and Adolscents Health, Health Services, Zahedan University of Medical Sciences, Zahedan, IR Iran

Surveying valuable and most recent information from internet, has become vital for researchers and scholars, because every day, thousands and perhaps millions of scientific works are brought out as digital resources which represented by internet and researchers can’t ignore this great resource to find related documents for their literature search, which may not be found in any library. With regard to variety of documents presented on the internet, search engines are one of the most effective search tools for finding information.

The aim of this study is to evaluate the three criteria, recall, preciseness and importance of the four search engines which are PubMed, Science Direct, Google Scholar and federated search of Iranian National Medical Digital Library in addiction (prevention and treatment) to select the most effective search engine for offering the best literature research.

Materials and Methods

This research was a cross-sectional study by which four popular search engines in medical sciences were evaluated. To select keywords, medical subject heading (Mesh) was used. We entered given keywords in the search engines and after searching, 10 first entries were evaluated. Direct observation was used as a mean for data collection and they were analyzed by descriptive statistics (number, percent number and mean) and inferential statistics, One way analysis of variance (ANOVA) and post hoc Tukey in Spss. 15 statistical software. P Value < 0.05 was considered statistically significant.

Results have shown that the search engines had different operations with regard to the evaluated criteria. Since P Value was 0.004 < 0.05 for preciseness and was 0.002 < 0.05 for importance, it shows significant difference among search engines. PubMed, Science Direct and Google Scholar were the best in recall, preciseness and importance respectively.

Conclusions

As literature research is one of the most important stages of research, it's better for researchers, especially Substance-Related Disorders scholars to use different search engines with the best recall, preciseness and importance in that subject field to reach desirable results while searching and they don’t depend on just one search engine.

1. Background

Surveying the valuable and the most recent information has become vital for researchers and scholars, because every day, thousands and perhaps millions scientific works are bought out as digital resources that represented by internet and researchers can’t ignore this great resource for literature review and they find related documents for their literature searche that may not be found in any library ( 1 ). Developing new technologies, scholars have faced abundant variety of collection literature research review with regard to subject of presentation and information space. Web cyberspace attracts the researchers’ attention because of variety of data with different forms and simultaneity. Literature search review is one of the most important search stages. Revising the literature search detected by search station in the broader tissue off helps researcher to expand the research view and his/her landscape. On the other hand he/she limits the research title and reaches to an intensive research question ( 2 ). Web, a huge resource of data plays an important role as an information reference for scholars. Nowadays, the rate of network data growth caused inaccessibility of information that is more worse than lacking them ( 3 ). With the creation and development of internet network as the greatest, the most varied and the most widespread information resource, simultaneous and various types of search tools are appeared to help users find information they want, these tools include search engines, meta-search engines and subject directories ( 4 ). Information retrieval is a challenge for users since search tools are too complex to navigate ( 5 ). As one of the most effective searching tools, its role is distinct because those search engines use software facilities which can find information from different sites for users. On the other hand, during the past decades developed knowledge in Substance-Related Disorders caught everyone’s eyes enormously ( 6 ). This growth is as a result of discussing about addiction as a national, regional and multi-dimensional problem. The volume of produced information and up to date personal data necessitates to study hours in a day, even a small part of this subject. Therefore, persuading to use internet is essential for updating data literature review. Then we concluded that the mentioned four popular search engines in medical sciences include: PubMed, Science Direct, Google Scholar and federated search of Iran’s health, treatment and medical education ministry and indicated that whether the of above mention search engines offered the best literature search about addiction (prevention and treatment). In Tober’s study, the four most popular search engines; PubMed, Science Direct, Scopus and Google Scholar are investigated to evaluate which search engine is the most effective for literature research in laser medicine. He concludes that all in all, Scopus was the most effective search engine in the literature research ,in case of requiring only an overview of the topic, even for a widespread and in depth investigation in area of life sciences and closely related topics, PubMed was more appropriate. Google Scholar, Science Direct are the best in preciseness and importance criterion in laser medicine ( 7 ). Also Bajpaie and et investigated in a research and compared 18 search tools, their results showed that four tools could be better than the others; these tools include two of the full-text scanners (High wire press & Google Scholar) and two citation scanners (PubMed & Scopus). The results show that, use of a single search tool can lead to loss of up to 70 % of the relevant citations in some cases. Hence, use of multiple search tools is recommended ( 8 ). Anders & Ivans compared PubMed with Google Scholar literature search in respiratory care topics by cross-sectional study and their research results showed that PubMed and Google Scholar had similar recall, but at precision criterion PubMed was better than the other. According to researches, PubMed was more efficient and better than Google Scholar with regard to the patient's searches and educational purposes ( 9 ).

In research by Flagas et al. a Comparison was made to evaluate the strength and weaknesses of PubMed, Scopus, Web of Science, and Google Scholar. Results showed that all databases were practical and offered numerous search facilities. PubMed and Google Scholar were free access for users. PubMed had offered optimal update frequency and include recent online articles; other database had rated articles based on some criteria such importance. For citation analysis, Scopus offered about 20% more coverage than Web of science; and Scopus covers the wider range of journals. Google Scholar can help in the retrieval of even the most oblique information and less often update ( 10 ). In a descriptive research, Mohammad Esmaeil, Lafzghazi and Gilvari compared six search engines and six meta-search engines in pharmaceutics information retrieval, results showed that if users survey in several search engines, they access to the relevant documents among the vastly available sources on web. Their research showed that Yahoo retrieved the most pharmaceutics documents and AOL had 62% precision and 21% recall, it retrieved the most relevant pharmaceutics documents. Among meta-search engines Dogpile was better than others ( 4 ). Outline description from evaluated search engines is explained in in the following:

1.1. PubMed

In 1997, PubMed (http://www.ncbi.nlm.nih.gov/PubMed) was offered by the National Library of Medicine on internet. It is one of the most popular and the most responsible resources on the World Wide Web for physicians and scholars ( 10 ). PubMed is a free search engine to search about medicine and biomedical journal literature. It searches several databases and interfaces Medline, directly. This search engine maps user’s search terms to the Medical subject heading (Mesh) and text words in Medline records and then searching ( 9 ). The PubMed offers users numerous powerful searche filters to limit their searches and gives them desirable retrieval information ( 11 ).

1.2. Science Direct

Science Direct (http://www.science direct.com) is a full text scientific database which is a part of the science verse and is provided by Elsevier publication in 1997. The web portal of Science Direct opens with features which invites the users just to browse the word scientific publications ( 7 ). This search engine is one of the greatest bibliographic and full text electronic collections about science, technology and medicine. Also we can have an exact searching with regard to limitations and abilities that is offered by Science Direct ( 12 ).

1.3. Google Scholar

Google Scholar (http://scholar.google.com): it’s design and handling is similar to that of Google search engine. It provides a simple way to search broadly for scholarly literature. In a particular place you can search across many disciplines and sources such as articles, theses, books, abstract and etc. This search engine helps you to find relevant scientific works in all over the world of sceince ( 12 ). The search results in Google Scholar can be limited to title, author, publication source, publishing date and other filters ( 13 ).

1.4. Integrated Digital Library (IDL):

IDL is an advanced system which gives us simple and one step access to all electronic resources at the Iranian National Medical Digital Library. Also we can browse and search all databases, e-journals, e-books and references in digital library alphabetically or by subject. Federated search allows you to search multiple online databases. Federated search saves your time and provides your favorite results quickly ( 14 ).

2. Objectives

The aim of this study is to evaluate the three criteria such as recall, precision and importance in the four search engines: PubMed, Science Direct, Google Scholar and federated search of Iranian National Medical Digital Library in addiction (prevention and treatment) to select the most effective search engine for offering the best literature research.

3. Materials and Methods

This research was cross-sectional study and we evaluated four famous and popular search engines in medical sciences include: PubMed, Science Direct, Google Scholar and federated search in IDL. We limited search to “Substance-related disorders” keyword in all search engines because this was the most common key word that specialists used. Documents that search engines offer are listed as results according to relevant value of query search with descent sequence. Hence, document offered as the first record in the search results, is the most related document to query search, from vision of that search engine ( 3 ). With regard to what was mentioned above and what cited to Tober’s study, sample research selected 10 first results of retrieval documents of each search engines. To compare search engines, three criteria, recall, precision and importance were evaluated ( 7 ). The calculation method for each criterion is explained as follow:

3.1. Performance test: Criteria of evaluation

The aim of the performance test was to get an overview about article dealing with the addiction topic (prevention and treatment). Therefore, we consulted with psychiatrist and selected proper key word among prevalent terms, and then matched with web version of mesh ( 15 ), and “Substance-Related Disorders” keyword was chosen as the most relevant query search at search engines in addiction (prevention and treatment).

1. The criterion ‘‘recall’’ (or ‘‘hits’’) is the number of found articles and is related to the integrity of the evidence base.

2. The criterion ‘‘precision’’ determines how well the filtered articles cover the topic of the search term and influences the time and cost of screening and the results for the related articles. It is evaluated by counting the search term in the search fields ‘‘title’’ (n title) and ‘‘abstract’’ (n abstract). The appearance of the terms in the field ‘‘title’’ (a) is two times more than the field ‘‘abstract’’ (b). Additionally the appearance is rated by the rank i, the position of the filtered article in the results list. From this the precision P can be calculated as follows:

P = ( (n + 1) – i ) • (a × n title + b × n abstract)

3. The criterion ‘‘importance’’ is determined by the number of citations “n” citation by publications of other authors. For this articles, citations that were in Science Direct and Google Scholar had been considered and for articles, citations were offered by PubMed and IDL and the other articles without citation, Web of knowledge citation database was used. Number of citation was used for calculating importance criterion and the rank i as described before. From this, the importance I is calculated as follows:

I = ((n + 1) − i) • n citation

For this study, only English written abstract and title which contain the search term “Substance-Related Disorders” were analyzed, for collecting information and data, directly observation was used. We entered keyword in each search engine, and to evaluate criteria, number of all retrieval articles (hits), titles, abstracts, and citations of 10 first articles in the result list were considered. All searches were conducted on November 29th, 2011 on Windows seven environment with Microsoft Internet Explorer in the Central Library of Zahedan Medical Sciences University. We used statistical software Spss 15 for data analysis. One-way analysis of variance (ANOVA) and post hoc Tukey was performed to evaluate search engines. P < 0.05 was considered statistically significant.

Results showed that PubMed retrieved most of the documents, 213 articles (32%) and it was better than the other search engines on recall criterion. Results from recall criterion were similar in Science Direct with 188 hits (28%) and, Google Scholar with 184 hits (27%). And IDL with 91 hits (13%) offered the least articles in this subject ( Table 1 ).

After survey of titles and abstract precision and statistical indicators of each search engines were investigated. Investigation of means showed that “the most precision” has allocated to Science Direct search engine with mean 25.2 (33%) and after that was ILD with mean 17.9 (24%), and then were PubMed with mean 16.2 (22%) and Google Scholar with mean 16 (21%). For detecting precision among search engines, One-way analysis of variance (ANOVA) was used and P value earned (0.726) which was more than 0.05, and this explained there was not statistically significant difference among search engines for precision. Whereas assessed P value from article titles investigation was P = 0.004 which showed significant difference in precision among search engines. Post hoc Tukey showed that there was difference between Science Direct with PubMed ( Table 2 ).

For importance, results showed that Google Scholar was the best and its means was 510.3 (82%), and after that was Science Direct, with mean 115.6 (18%). IDL with mean 0.3 (0%) and PubMed were not scored. Results of One-way analysis of variance (ANOVA) showed P Value = 0.002 that was less than 0.05, so we used Post hoc Tukey. Finding from importance criterion showed statistically significant difference between Google Scholar with each of the others search engines ( Table 3 ).

5. Discussion

Literature search is one of the most important steps in the research process. We review literature to avoid heavy works, to ensure we have a thorough understanding of the topic, to identify similar work done within the area, to identify knowledge gaps that demand further investigation, to compare previous findings, to critique existing finding and suggest further studies ( 16 ), which this features increase the importance or the exact literature review. Review of literature accomplishes for literature search in medical sciences on the internet by different search engines that they were compared, evaluated and reviewed by the other researchers in different countries to identify their search’s features and abilities ( 5 - 10 ) ( 15 - 22 ). This research was done to select the most effective search engine for offering the best literature research in addiction (prevention and treatment).

Investigations showed that search engines in various subject fields with regard to unique abilities and facilities are done differently, which this affair supported by other researchers, too ( 16 , 18 ). Search engines cover just a limited part of accessible information on the web and neither of them don’t have total recall and precision. The scale of recall, precision ( 23 ), and importance of search engines are different with regard to purpose of researchers from search. Results of this study showed that evaluated criteria were different in search engines, whereas PubMed had the most recall criterion and then were Science Direct and Google Scholar with similar recall. Recall in PubMed was about two times more than ILD’s recall. The results of Tober study were different, whereas PubMed had least recall in laser medicine ( 7 ). At the other research was done in respiratory care, results showed that the function of two search engines (PubMed & Google Scholar) was similar in recall criterion ( 9 ), our study showed this similarity between Google Scholar and Science Direct.

Means investigation explained that Science Direct offered the best precision and related documents in addiction. Although Results of One - way analysis of variance (ANOVA) did not show significant difference in precision among search engines, results were different in investigation of retrieved articles titles which P - value was less than 0.05 that explanatory existence showed significant difference among search engines, post hoc Tukey showed that considerable difference exists between Science Direct and PubMed. Also, in Tober study, Science Direct after Scopus had the best precision 7. But in the other research, results were different whereas PubMed had above precession ( 9 ).

In this study, Google Scholar had maximum importance and the next was Science Direct. PubMed and IDL were not scored. In PubMed Tober’s study, results were similar to our study in this criterion. For importance, results of ANOVA showed significant difference, moreover post hoc Tukey and comparing means showed that the difference was remarkable among Google Scholar. Also, former researcher supported indication of Google Scholar for offering the citations which caused importance of articles ( 10 , 20 , 22 ). Reviews of this research showed that IDL did not offer acceptable results in any of three criteria, with regard to expensive cost that Iran’s health, treatment and medical education ministry pay for funding medical sciences universities and this will be revision to buy and use this database or be changed the operation of federated search.

We had local limitation for our searches in federated search of IDL and Science direct because this search engines were not available to the public. The other limitation to our study was that we only used the “Substance- related disorders” keyword in search engines and didn’t use the related keywords. Next limitation was that at first, we wanted to evaluate Scopus along with the other search engines, but Scopus was disconnecting in Iran and thus excluded from this study. Results showed that use of one search engine for literature review in addiction (prevention and treatment), neither proper nor search engines can’t give us the best results. Former studies, also recommended combined search and simultaneous use from several search engines to reach effective and related results ( 4 , 8 , 17 ). With regard to broad extent, multilateral addiction subject and new sights discussion, it is recommended that researchers must use different search engines with the best recall, precision and importance to reach the best results, they must continue their search with abilities and various services that search engines give.

Acknowledgments

We appreciate Mr. Dashipoor for statistical analysis.

Implication for health policy/practice/research/medical education:

The aim of this study is the evaluation of three criteria, recall, precision and importance in the four search engines: PubMed, Science Direct, Google Scholar and federated search of Iranian National Medical Digital Library in addiction (prevention and treatment) to select the most effective search engine for offering the best literature research.

Please cite this paper as:

Samadzadeh GR, Rigi T, Ganjali AR. Comparison of Four Search Engines and their efficacy With Emphasis on Literature Research in Addiction (Prevention and Treatment). Int J High Risk Behav Addict.2013; 1(4): 166-71. DOI: 10.5812/ijhrba.6551

Authors’ Contribution:

Reza Samadzadeh conceived of and designed the study. Tahereh Rigi analyzed and interpreted the data. Alireza Ganjali drafted the paper critically revised it for important intellectual content. All authors gave final approval of the version to be publish.

Financial Disclosure:

Who have no relevant financial interests are asked to provide a statement indicating that they have no financial interests related to the material in the manuscript.

Funding/Support:

This study was no supported by Health or teaching and research Institutes.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Open access
• Published: 25 February 2011

In search of the right literature search engine(s)

• Akhilesh Bajpai 1 ,
• Sravanthi Davuluri 1 ,
• Haritha Haridas 1 ,
• Greta Kasliwal 1 ,
• Deepti H 2 ,
• Sreelakshmi KS 1 ,
• Darshan Chandrashekar 2 ,
• Pranami Bora 1 ,
• Mohammed Farouk 1 ,
• Neelima Chitturi 1 ,
• Samudyata V 2 ,
• ArunNehru KP 3 &
• Kshitish Acharya 2  

Nature Precedings ( 2011 ) Cite this article

1382 Accesses

9 Citations

3 Altmetric

Metrics details

Collecting scientific publications related to a specific topic is crucial for different phases of research, health care and ‘effective text mining’. Available bio-literature search engines vary in their ability to scan different sections of articles, for the user-provided search terms and/or phrases. Since a thorough scientific analysis of all major bibliographic tools has not been done, their selection has often remained subjective. We have considered most of the existing bio-literature search engines (http://www.shodhaka.com/startbioinfo/LitSearch.html) and performed an extensive analysis of 18 literature search engines, over a period of about 3 years. Eight different topics were taken and about 50 searches were performed using the selected search engines. The relevance of retrieved citations was carefully assessed after every search, to estimate the citation retrieval efficiency. Different other features of the search tools were also compared using a semi-quantitative method.

The study provides the first tangible comparative account of relative retrieval efficiency, input and output features, resource coverage and a few other utilities of the bio-literature search tools. The results show that using a single search tool can lead to loss of up to 75% relevant citations in some cases. Hence, use of multiple search tools is recommended. But, it would also not be practical to use all or too many search engines. The detailed observations made in the study can assist researchers and health professionals in making a more objective selection among the search engines. A corollary study revealed relative advantages and disadvantages of the full-text scanning tools.*Conclusion*While many studies have attempted to compare literature search engines, important questions remained unanswered till date. Following are some of those questions, along with answers provided by the current study:a) Which tools should be used to get the maximum number of relevant citations with a reasonable effort? ANSWER: _Using PubMed, Scopus, Google Scholar and HighWire Press individually, and then compiling the hits into a union list is the best option. Citation-Compiler (http://www.shodhaka.com/compiler) can help to compile the results from each of the recommended tool._b) What is the approximate percentage of relevant citations expected to be lost if only one search engine is used? ANSWER: _About 39% of the total relevant citations were lost in searches across 4 topics; 49% hits were lost while using PubMed or HighWire Press, while 37% and 20% loss was noticed while using Google Scholar and Scopus, respectively._c) Which full text search engines can be recommended in general? ANSWER: HighWire Press and Google Scholar. d) Among the mostly used search engines, which one can be recommended for best precision? ANSWER: EBIMed. e) Among the mostly used search engines, which one can be recommended for best recall? ANSWER: Depending on the type of query used, best recall could be obtained by HighWire Press or Scopus.

Article PDF

Author information, authors and affiliations.

Institute of Bioinformatics and Applied Biotechnology (IBAB), Biotech Park, Electronic City, Bangalore, 560100, Karnataka state, India

Akhilesh Bajpai, Sravanthi Davuluri, Haritha Haridas, Greta Kasliwal, Sreelakshmi KS, Pranami Bora, Mohammed Farouk & Neelima Chitturi

Institute of Bioinformatics and Applied Biotechnology (IBAB), Biotech Park, Electronic City, Bangalore - 560100, Karnataka state, India, www.ibab.ac.in / Shodhaka Life Sciences Pvt. Ltd., IBAB, Biotech Park, Electronic City, Bangalore, 560100, Karnataka

Deepti H, Darshan Chandrashekar, Samudyata V & Kshitish Acharya

Shodhaka Life Sciences Pvt. Ltd., IBAB, Biotech Park, Electronic City, Bangalore, 560100, Karnataka state, India

ArunNehru KP

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Kshitish Acharya .

Rights and permissions

Creative Commons Attribution 3.0 License .

Reprints and Permissions

About this article

Cite this article.

Bajpai, A., Davuluri, S., Haridas, H. et al. In search of the right literature search engine(s). Nat Prec (2011). https://doi.org/10.1038/npre.2011.2101.3

Download citation

Received : 25 February 2011

Accepted : 25 February 2011

Published : 25 February 2011

DOI : https://doi.org/10.1038/npre.2011.2101.3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• literature search
• text mining
• search engines
• search tools
• search programs
• publication search
• Highwire Press
• google scholar
• citation retrieval
• literature mining
• bibliography
• publications

This article is cited by

Go2pub: querying pubmed with semantic expansion of gene ontology terms.

• Charles Bettembourg
• Christian Diot
• Olivier Dameron

Journal of Biomedical Semantics (2012)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Personal Finance
• Policy & Politics

The Role of Search Engine in Research Journey

Dr. Anand Shankar Raja M , September 10, 2022,  0 Comments

To facilitate the entire process the basic requirement is a well-planned and well-phased search process and a suitable search engine. Hence, this article provides a holistic view of the search process and associated elements in conducting a quality review of the literature and other research endeavors.

Read also: The importance of using Citations & References in Thesis & research papers

In addition, search engines play a major role to facilitate the entire process. Not just to conduct a literature review, but also to find research articles for different purposes. Different databases have different features and serve different purposes. Hence the researchers need to go through the entire search process. Apart from research, as academicians, the thirst for knowledge is always unquenched and hence all scholars must make the best use of search engines to acquire knowledge, share knowledge, and transfer knowledge to the other scholarly community.

Introduction

Searching for the right article to conduct a review of literature is very important and is the primary goal of research. It is not very easy to identify the right article for the review of literature because articles cannot be randomly collected from different databases. There should be a common thread between the articles being used in the literature review. In addition, literature reviews are a daunting challenge for researchers especially when they conduct research in a few unexplored areas.

Most of the white reports which are published by various ministries, corporate organisations, and standalone platforms appear on the website. It consumes more time to locate the right articles because of the unavailability of search strategies. Omitting various white reports and not including them in the research work may not help the researcher produce a quality article as it does not focus on the comprehensive approach. Literature reviews should be very comprehensive and should help the researchers in several ways. Researchers should be able to find the research gap through the literature review and establish something new and meaningful to different stakeholders. In addition, the researchers will be able to define the research objectives, plan the scope of the study and introspect the pros and cons of the research. To facilitate the basic requirements, it is important to follow certain search designs and approaches. Hence, the process of conducting a literature review is essential beyond the boundaries of just conducting a research study. Researchers must systematically synthesise the information to arrive at meaningful conclusions and to introspect contrasting viewpoints of previous studies. However, there is no set of defined rules to organise literature reviews for a research article.

Literature review takes different forms depending on several factors such as nature of research, area of research, Articles search strategy, inclusive and exclusion factors, availability of articles, nature of documents, etc. Since there are no well-defined parameters to conduct the literature reviews there is no uniformity, transparency, visibility, or ethics. Literature reviews are a mere summary of all the collected articles. However, the impact of literature review in enhancing the quality of a research article is not realised by the researchers. Literature reviews are not just a summary of randomly collected articles from different sources but a meaningful selection of documents that are collected, evaluated and summarised to arrive at meaningful conclusions. Researchers should understand the importance of literature reviews as it helps them make valuable contributions to a particular field of study says (Winchester & Salji, 2016). Researchers can adopt a less formal approach for scanning the literature sources keeping in mind a standard set of parameters. However, the parameters considered to scrutinize the articles have to be holistic without being biased. This will enhance the richness of information and will boost the visibility of the article. Inappropriate literature will be an accumulation of information from different sources but will not have the fibre to identify the research gap. Perhaps the main problem in positing a research gap is the selection of appropriate literature. Locating the research gap from literature is a systematic process. The gap does exist not only in articles published in journals but also in the white reports published by corporates and Government institutions. These reports are highly knowledge-driven, socially relevant, and updated in terms of facts and figures. Indeed, they are not biased about population and data collection. A failure to consider these reports will lower the quality of research. Hence, research gaps spotted through literature reviews should consider various sources which are put together to be critically evaluated and summarised. Long gone are days when the research was very traditional and systematically done in an old fashion. Modern research thoughts are dynamic and need based. It intends to serve its purpose and creates an impact on society and the economy. Thus, research materials and methods have become more agile and resilient depending on the need. Systematically searching articles for conducting the review of literature should involve transparency in the entire process. This will help the researchers incorporate a systematic approach to article search, filtering, and reporting process.

There are a few popular databases that are widely used by researchers to find articles for their respective research purposes. However, the search criteria are not common among all these databases. Different databases have different parameters as a default filter option. Some are flexible and allow the researcher to use multiple search options and filter options, but few databases are rigid and are confined only to limited parameters.

Moreover, grey literature reviews published by standalone companies and government agencies are avoided because of the unavailability of the search and filtering parameters. A comprehensive literature review cannot depend only on popular databases but should also consider a wide range of databases including the databases for grey articles and reports. Moreover, consideration of multiple databases will help in avoiding publication bias and will help the researchers to widely get exposed to many hidden ideas and knowledge.

Thus, this article aims to first provide guidelines for a systematic search of scholarly articles, define and list down inclusion criteria and exclusion criteria depending on the nature of databases and finally chalk out a few strategies for the researchers to avoid article selection bias. Thus, the objective and focus of the chapter are to propose guidelines and strategies for literature search and selection for transparent and unbiased reporting. Few popular databases such as Google Scholar, Emerald, Taylor & Francis, Elsevier, ProQuest, and IEEE have been considered. In addition, the Social Sciences Evidence-Informed Decision-Making (EIDM) approach plays an important role because research aims to benefit multiple stakeholders and hedge toward decision-making. The EIDM process also involves systematic and rigorous research of articles to synthesise the evidence. Thus, the secondary objective of this chapter is to highlight the importance of the EIDM framework exclusively for Social Science research. From this research, it is possible to understand the existing bias in the article search process, data selection, and screening and to contribute to the research community a holistic framework that explains appropriate search strategies, inclusion and exclusion criteria, and strategies to avoid information redundancy in the entire process.

Importance of search engine

The quality of research depends on several factors and one of the most essential ones is the rich collection of knowledge, facts, and figures. Every researcher has an intention to produce something new for the world and with this intention, he/she searches for wide resources from various databases. Higher Educational Institutions having a library have taken efforts to scan and store the resources in the digital format which is available to researchers across the world. In addition, there are many famous databases such as Scopus, Web of Science, PubMed, ERIC, IEEE, ScienceDirect, DOAJ, etc. which have huge volumes of resources. Researchers depend on these databases to collect articles based on their needs and expectations. Whether starting with a research project or dealing with doctoral research it is a must to enrich knowledge by reading articles published by various other researchers. For this process, there is a felt need to depend on multiple databases. The development of search engines has helped researchers in multiple ways. Research work is enriched with quality when it showcases something new and interesting. To identify the research gap and to narrow down the research scope it is necessary to find appropriate articles and data resources for which search engines are useful. However, researchers must know the art of using the right keywords to identify the right data resources according to their interests and needs.

Searching the pioneering authors

Before starting with a literature review it is important to systematically search for articles that will fit into the scope of the study. First and foremost, it is important to identify a few pioneers in a particular field who have distinguished themselves by finding a profound theme, theory, model, methodology, etc. This will help the researchers find the original article which would be available in any of the databases. In addition, the original idea of the pioneering researcher can be traced to understand the original piece of work. This process can be facilitated by searching the articles using the founding authors’ names. In addition, this search will also help in knowing the definition of a concept or a term by systematically investigating the original work. As time progresses a concept will be modified according to the situation and hence the originality would be lacking. Thus, searching the original work and introspecting the same will give the researcher a fresh opinion and will help them trace the growth of a concept over years. Hence, in the search methodology, the researchers should first know about the pioneering authors and search accordingly.

Perl growing or snowballing

In the search process, researchers should also give importance to the Perl growing approach which is also known as citation searching or citation tracking. Few good research articles can be found using this approach which is less time-consuming but needs patience because it uses different parameters to locate the article. Article title, Journal name, DOI number, Author details, and year of publication are the primary factors that are used. This method is more suitable for conducting a systematic literature review. The researchers can depend on Science Citation Index (SCI), Social Science Citation Index (SSCI), Arts and Humanities Citation Index (AHCI), Conference Proceedings Citation Index (CPCI), Book Citation Index (BCI), Emerging Source Citation Index (ESCI), Data Citation Index (DCI). Instead of using a predefined search strategy, this method helps in the easy identification of the relevant sources.

Google Books and Google Scholar

Academic search engines have gained popularity in recent times as researchers have started to use methodologies that require extensive usage of articles to be used in the review of literature chapters. Hence, research scholars are now dependent on different search engines to find suitable articles for their research. One of the search engines which remain unused is google books and Microsoft academic. On the other hand, in recent times research gateways and academia have become popular. Google books are a combination of self-published books, publisher-supplied, etc. In addition, google books are classified into four categories which include: full view, preview, snippet view, and no preview. Research scholars can use the full view option to find books of their choice that are available in the public domain. There are also a few rare collections of books that are available in google books however only a few pages are available for view. In this case, researchers can purchase those books if needed. Google books also provided advanced search options for the readers to exactly locate the resources that they expect. The advanced search option includes filter options such as search, content, language, title, author, publisher, subject, publication date, ISBN, ISSN, etc. This advanced feature with flexible filter options will reduce search time and effort.

The second most well-known search engine is Google scholar which helps in finding scholarly articles based on three popular matrices. Google scholar has a huge collection of various resources. This also helps in finding grey literature, which is a collection of government reports, organisations’ white papers, and all the other articles and reports which are produced out of the commercial or academic arena. Though grey literature sources are avoided from being cited in research articles they are rich in information and provide extensive knowledge to the readers. Moreover, google scholar is not very much narrow focussed and does not limit the search hence researchers who are beginning their research career can start exploring google scholar for various purposes. Since google scholar relies on keywords for search and not the metadata researchers can use different search terms to scrutinise the articles.

Research gate for discovering research and staying connected with the scientific community

Research Gate is a platform for researchers to connect with other world researchers for the scientific search of knowledge, share research articles, ask questions and clear doubts and extend research collaborations. Since every researcher across the world is taking efforts to showcase their research profile they update and maintain a clear profile. Moreover, each researcher is given a profile score and shows the number of readers that a particular has fetched. It gives readers the confidence to depend on certain documents to be used in their research. Altmetric for researchers Researchers across the world are interconnected with each other with the help of Information Technology. Various research documents are uploaded to different databases and academic portals. Academic, and ResearchGate are all a few examples where researchers comment, follow, request, post, read and extend research collaborations with other researchers. Altmetric gathers information from different sources where research is being discussed. The data is collected from research forums, research blogs, documents, white reports, social media, etc. All the hidden information is well pooled together and the same is made available to the world researchers. The journals and collections tab helps researchers know the best performing journal in terms of audience outreach. This feature will help the researchers identify the maximum reads fetched by an article and its outreach. Almetrics will help the researchers find the most popular articles which can be further used to conduct a review of the literature.

Microsoft academic

Microsoft Academic is also a platform and a database that allows researchers to showcase their research publications to the outside world. It encourages researchers to upload their research publications and helps them unleash the power of knowledge sharing and knowledge acquisition. There were millions of publications available with Microsoft academics. In the year 2021, this database was closed and was taken over by OurResearch. However, researchers can use OurResearch platform to search and locate the best data as per their needs.

Conclusion Search engines play a major role in the life of each researcher. Search engines are of different types and have different features. There are more than 100 search engines available for researchers. These search services, if used correctly, will help the researchers find the best resource which can be used in their research work. However, not all search engines are useful. Few may not be as per the researchers’ expectations; hence the researchers must use filter process, and inclusion and exclusion criteria to finalise the resources they need.

Tags: research purpose , researchers , review of literature , scholars , search engine , search strategy

Become a contributing writer

Me interview series: west’s idea of globalization, developing countries, and their relevance.

ME Interview Series on Globalization, West &  Developing Countries with Hervé Azouloy, President of ATHES Finance and Participation  & Ezilarsan PKP, founder...

VE Idea Accelerator Series: MSME Clusters in the State of Gujarat – An Introduction

Skills & Work: Reskilling Employability Needs for Post-Covid World

Women Entrepreneurs Breaking Gender Biases

2021 of Change Management – What we did differently?

Analysis of Electric Vehicle (EV) in India

Health Prime Rider: Here’s Why You Must Get a Rider that Covers Your Doctor Consultation Fee

Health is one of the things most important things in your life. Without health, all the wealth you ever earn will...

Mosambi (sweet lemon) & Weight loss: 6 Benefits of Mosambi Juice

Women, Mobility & Shakti Scheme –Opportunities And Challenges

My maid came early this morning and told me she wouldn’t be coming for the next three days. I was astonished...

Short Stories: Japan & Rising Sun -‘Wow’! It’s a dream that came true

Eduexpress.in | Design thinking, kids & how to create a nurturing environment?

Judging others is often based on our own ignorance!

Indian students studying abroad, India & Recession

Experts are constantly warning that countries such as the United States and the United Kingdom may face a recession amid the...

Aggressive vs. Discrete Marketing: What Works Best for Generation Z

Connect with us on Facebook

Information is key but not equal to expertise possessed or the ability to do things in a professional way, co-founders ( required ), equity with no monthly pay…, multimillion dollar idea & cofounder pay package.

• Mobile & Apps
• About MarketExpress
• Advertise with us
• Terms and Conditions
• MarketExpress Sitemap
• Deutsche Welle
• Capital Cube

MarketExpress Media & Education

MarketExpress © 2023 All Rights Reserved

Powered by J Simple Solutions Made in Mumbai

10 Open Science tools for literature review you should know about

Here are 10 literature search tools that will make your scientific literature search faster and more convenient. All of the presented literature review software is free and follows Open Science principles.

Traditionally, scientific literature has been tucked away behind paywalls of academic publishers. Not only is the access to papers often restricted, but subscriptions are required to use many scientific search engines. This practice discriminates against universities and institutions who cannot afford the licenses, e.g. in low-income countries. Closed publishing also makes it hard for persons not affiliated with research institutes, such as freelance journalists or the public, to learn about scientific discoveries. 

The proportion of research accessible publicly today at no cost varies between disciplines . While in the biomedical sciences and mathematics, the majority of research published between 2009 and 2015 was openly accessible, this held true only for around 15 percent of publications in chemistry. Luckily, the interest in open access publishing is steadily increasing and has gained momentum in the past decade or so.

Many governmental funding bodies around the world nowadays require science resulting from grant money they provided to be available publicly for free. The exact requirements vary and UNESCO is currently developing a framework that specifies standards for the whole area of Open Science. 

Once I started my research on the topic, I was astonished by just how many free Open Science tools for literature review already exist! Read on below for 10 literature search tools — from a search engines for research papers, over literature review software that helps you quickly find open access versions of papers, to tools that help you save the correct citation in one click.

Tools for Literature review

First, an overview of the literature search tools in this blog post:

ScienceOpen

Citation Gecko

Local Citation Network

ResearchRabbit

Open Access Button

EndNote Click

Read by QxMD

I divided the tools into four categories:

Search engines for research papers

Literature review software based on citation networks

Locating open access scientific papers, and

Other tools that help in the literature review

Here, we go!

The best place to start a scientific literature search is with a search engine for research papers. Here are two you might not have heard of!

Want to perform a literature search and don’t want to pay for Web of Science or Scopus or perhaps you are tired of the limited functionality of the free Google Scholar ? ScienceOpen is many things, among others a search engine for research papers. Despite being owned by a private company, this scientific search engine is freely accessible with visually appealing and functional design. Search results are clearly labelled for type of publication, number of citations, altmetrics scores etc. and allow for filtering. You can also access citation metrics, i.e., display which publications have cited a certain paper.

Recommended by a reader of the blog (thank you!), the Lens is a search tool that doesn’t only allow you to search the scholarly literature but patents too! Millions of patents from over 95 jurisdictions can be searched. The Lens is run by the non-profit social enterprise Cambia. The search engine is free to use for the public, though charges occur for commercial use and to get additional functionality.

Literature Review software based on citation networks

The next category of tools we will be looking at are a bit more advanced than a simple search engine for research papers. These literature search tools help you discover scientific literature you may have missed by visualising citation networks.

Citation Gecko 

The literature search tool Citation Gecko is an open source web app that makes it easier to discover relevant scientific literature than your average keyword-based search engine for research papers. It works in the following way: First you upload about 5-6 “seed papers”. The program then extracts all references in and to these seed papers and creates a visual citation network. The nodes are displayed in different colours and sizes depending on whether the papers are citing a seed paper or are cited by it and how many, respectively. By combing through the citation network, you can discover new papers that may be relevant for your scientific literature search. You can also increase your citation network step by step by including more seed papers. 

This literature review tool was developed by Barney Walker , and the underlying citation data is provided by Crossref and Open Citations .

Local Citation Network 

Similar to Citation Gecko, Local Citation Network is an open source tool that works as a scientific search engine on steroids. Local Citation Network was developed by Physician Scientist Tim Wölfle. This literature review tool works best if you feed it with a larger library of seed papers than required for Citation Gecko. Therefore, Wölfle recommends using it at the end of your scientific literature search to identify papers you may have missed. 

As an alternative to the literature search tools Citation Gecko and Local Citation Network, a reader of the blog recommended ResearchRabbit . It’s free to use and looks like a versatile piece of literature review software helping you build your own citation network. ResearchRabbit lets you add labels to the entries in your citation network, download PDFs of papers and sign up for email alerts for new papers related to your research topic. Instead of a tool to use only once during your scientific literature search, ResearchRabbit seems to function more like a private scientific library storing (and connecting) all the papers in your field.

Run by (former) researchers and engineers, ResearchRabbit is partly financed through donations but their website does not state where the core funding of this literature review software originates from.

Locating open access scientific papers

You may face the problem in your scientific literature search that you don’t have access to every research paper you are interested in. I highly recommend installing at least one of the open access tools below so you can quickly locate freely accessible versions of the scientific literature if available anywhere.

Open Access Button 

Works like the scientific search engine Sci-hub but is legal: You enter the DOI, link or citation of a paper and the literature review tool Open Access Button displays it if freely accessible anywhere. To find an open access version, Open Access Button searches thousands of repositories, for example, preprint servers, authors’ personal pages, open access journals and other aggregators such as the COnnecting REpositories service based at The Open University in the UK ( CORE ) , the EU-funded OpenAire infrastructure, and the US community initiative Share . 

If the article you are looking for isn’t freely available, Open Access Button asks the author to share it to a repository. You can enter your email address to be notified once it has become available. 

Open Access Button is also available as browser plugin, which means that a button appears next to an article whenever a free version is available. This search engine for research papers is funded by non-profit foundations and is open source. 

Unpaywall 

Unpaywall is a search engine for research papers similar to Open Access Button — but only available as browser plugin. If the article you are looking at is behind a paywall but freely accessible somewhere else, a green button appears on the right side of the article. I installed it recently and regret not having done it sooner, it works really smoothly! I think the plugin is a great help in your scientific literature search.

Unpaywall is run by the non-profit organisation Our Research who has created a fleet of open science tools.

EndNote Click 

Another browser extension that lets you access the scientific literature for free if available is EndNote Click (formerly Kopernio). EndNote Click claims to be faster than other search engines for research papers bypassing redirects and verification steps. I personally don’t find the Unpaywall or Open Access Button plugins inconvenient to use but I’d encourage you to try out all of these scientific search engines and see what works best for you. 

One advantage of EndNote Click that a reader of the blog told me about is the side bar that appears when opening a paper through the plugin. It lets you, for example, save citations quickly, avoiding time-consuming searches on publishers’ websites. 

As the reference manager, EndNote Click is part of the research analytics company Clarivate.  

Other tools for literature review

This last category of literature search tools features a tool that creates a personalised feed of scientific literature for you and another that makes citing the scientific literature effortless.

Available as an app or in a browser window, the literature review tool Read lets you create a personalised feed that is updated daily with new papers on research topics or from journals of your choice. If there is an openly accessible version of an article, you can read it with one click. If your institution has journal subscriptions, you can also link them to your Read profile. Read has been created by the company QxMD and is free to use. 

CiteAs 

You discovered a promising paper in your scientific literature search and want to cite it? CiteAs is a convenient literature review tool to obtain the correct citation for any publication, preprint, software or dataset in one click. Funded by the Alfred P. Sloan Foundation, CiteAs is operated partly by the non-profit Our Research . 

Beyond literature review tools

There you have it, 10 tools for literature review that are all completely free and follow Open Science principles.

Of course, finding a great literature review tool, such as a search engine for research papers or a citation tool, is only one essential part in the whole process of writing a scientific paper. If you would like to learn a complete process to write a scientific article step by step, then you’ll love our free training. Simply click on the orange button below to watch it now (or sign up to watch it later).

The Power of Search. Search Engines as Key Marketing Parameters

A systematic literature review, research paper (postgraduate), 2018, 62 pages, grade: 1,7, benjamin chée (author), table of contents, 1. introduction, 2. conceptual framework.

3. METHODOLOGY 3.1. Database Search Query 3.2. Summary and Categorization

4. EMPIRICAL FINDINGS 4.1. Analysis and Classification of Research Articles 4.2. Search Engines 4.2.1. Search Engine Usefulness, Utilization, and User Experience 4.2.2. Search Engine Profitability 4.2.3. Type of Advertisement Auctions 4.2.4. Organic Page Rank 4.3. Advertisers 4.3.1. Paid Search Profitability and Effectiveness 4.3.2. Bidding Strategies and Keyword Competition 4.3.3. Ad Position Performance 4.4. Consumers

5. DISCUSSION 5.1. Theoretical Contribution 5.2. Practical Contribution 5.3. Limitations 5.4. Future Research Implications

6. CONCLUSION

List of Figures

Figure 1. Conceptual Framework of Search Engine Research

Figure 2. Number of Relevant Articles per Year

Figure 3. Number of Relevant Articles per Topic

List of Abbreviations

Abbildung in dieser Leseprobe nicht enthalten

This paper examines the academic research that has been published during the past 20 years concerning search engine marketing. It classifies the most prominent papers and establishes a conceptual framework consisting of three broad areas and 20 different research themes. This is the first literature review of search-engine-related research, proposing a comprehensive conceptual framework.

The purpose of this study is to conceptualize past research on search engines, develop a framework of topic areas and their relationships among each other, and identify research gaps and questions for future research.

Research Design:

The literature review covers all academic research papers from five acclaimed journals of the past 20 years that match predefined keywords and deal with search engines as parameters in modern-day marketing.

Originality and Value:

This literature review of search engine research is the first of its kind, creating a framework of research fields exclusively for online search. Furthermore, it presents the related academic work in an interdependent structure of topics and suggests future research fields.

Keywords: search engine, paid search, keyword auctions, digital marketing, online marketing

Digital marketing, social media marketing and mobile marketing are an important part of virtually every business’ marketing strategy. For the time period from 1987 to 2015, researchers have carried out literature reviews (Lamberton & Stephen, 2016; Ngai, 2003; Schibrowsky, Peltier, & Nill, 2007) that gave overviews of the evolution of digital marketing research as well as current trend topic areas. It is remarkable that until now no literature review has focused on the topic of search engine marketing alone, given its prominent role in modern-day online business practice. The prevailing relevance of search engine marketing can be witnessed in the European Commission’s recent decision in July 2018 to impose a record antitrust fine of €4.34 billion on Google for abusing its market power (Satariano & Nicas, 2018). As the owner of the market leading Smartphone operating system Android, “Google has used Android as a vehicle to cement the dominance of its search engine”, as stated by competition policy commissioner Margrethe Vestager (European Commission, 2018). The previous record antitrust fine of €2.4 billion in June 2017 was imposed on Google as well – also related to unfair search engine competition (Scott, 2018). The European Commission’s rigorous measures against the world’s leading search engine provider and the record-breaking amount of the fines, which were calculated based on Google’s paid search revenue from Android devices in the European Economic Area, show the enormous influence that search engines have on electronic commerce today (European Commission, 2018).

Search engines’ major impact on not only businesses but also economies necessitates a comprehensive overview of past literature.

The latest literature review in the field of digital, social media, and mobile marketing was carried out by Lamberton and Stephen (2016), who built onto the two previous internet marketing literature reviews conducted by Ngai (2003) and Schibrowsky et al. (2007). They noticed that despite the importance and long existence of search engines as a research field, relatively few citations in economics literature can be observed. For example, the topic “search engines” only showed 3.4 mean cites per year from 2001 to 2015.

They further argue that despite a large body of research, no comprehensive answers have been researched to fundamental questions such as “What is the optimal balance between online and offline marketing?” or “What makes a digital marketing initiative a success for firms or consumers? Are there metrics beyond ROI that matter?” (Lamberton & Stephen, 2016, p. 166); questions that should also be considered in research about search engine marketing.

It is particularly interesting to examine how the different research topics, which were identified as emerging digital marketing research topics by Lamberton and Stephen (2016), are covered in search engine articles. These topics included collective behavior, online and offline crossover, as well as regulation and digital privacy issues. Additionally, it needs to be discussed if the main categories that Schibrowsky et al. (2007) found will also be applicable to search engine research: Consumer behavior, business to business, strategy, communication, product/brand, distribution, pricing, research issues, political legal, education. It seems that these categories are too general to create additional value for a search-engine-specific research framework.

This leads to this review’s guiding research question: Which elements of search engine advertising play a role in academic research and what is their relationship to each other? As part of this investigation, it should also be considered which aspects have been studied extensively and which areas should be examined more thoroughly.

Hence, the purpose of this work is not to provide a comprehensive literature review of every article related to search engines, as Lamberton and Stephen (2016) already pointed out. The scope of this work is simply too limited. The real contribution will be the identification of common research areas, labelling of search engine research fields and uncovering the underlying connections between the research topics.

Future research will benefit from this structured overview of academic search engine research. It will make it easier to categorize and locate a research article within the search engine area. Blind spots in the collected past literature are revealed and suggested for future studies.

The methodological approach is similar Ngai’s (2003) work, where he focused his search for relevant internet marketing literature on three publication areas, namely marketing, general business management, and information technology. He sorted the resulting articles into broad categories and then segmented them into more specific subcategories. Subsequently, he analyzed the distribution of the years of publication and the distribution of articles by subject.

The difference in this literature review is that mostly marketing journals were considered, due to the scope of this work and the slight results in the business management area.

Past research of search engine marketing involves at least one of the following three market participants, from now on referred to as agents: search engines, advertisers, consumers.

The analysis of the 52 articles that matched the search query resulted in the identification of 20 different research subtopics. The subtopics are pictured as rectangular boxes (see Figure 1. Conceptual Framework of Search Engine Research, p. 10). These research subtopics can be collocated according to their connection to search engines, advertisers or consumers.

Half of the subtopics could be distinctlyassigned to one of the agents, for example “Paid Search Profitability”. Although “Paid Search Profitability” is obviously influenced by consumers and search engines as well, it is primarily dependent on the advertisers’ marketing mix, advertising budget, search engine advertising strategies, internal cost accounting and many other factors. This logic was applied to all subtopics: Whenever a topic was primarily dependent on a single agent, it was assigned to that specific area. In the framework illustration, the agent areas are all connected with double-headed arrows to emphasize their interdependence.

The remaining ten subtopics that could not be attributed to one of the three agents were positioned in between two agents, illustrated as a grey-shaded area. All subtopics in one grey area have in common that they are heavily dependent on not one but two agents. “Paid Search Effectiveness”, as an example, cannot be explained by solely analyzing advertisers or search engines. Each agent contributes their part to the effectiveness; advertisers might need to create engaging ads, and search engines should display them to the right people.

This visualization uncovers the overall emphasis on advertiser-related marketing research and simultaneously highlights the lack of consumer-centric research. Needless to say, academic marketing research has focused on a plethora of consumer-centered topics. However, in this context, consumer-centric research means topics that are search-engine-related and analyze the consumer perspective of it.

Naturally, advertisers and consumers are strongly connected, and all advertiser subtopics involve consumers as well. However, purely consumer-oriented matters have rarely been considered. Topics directly related to consumers were consumer trust in search engine results or advertisements, and consumers’ price sensitivity.

The presented conceptual framework of search engine research is the first attempt of structuring this academic research area. It raises no claim to completeness; the arranged topics were merely a result of this literature review. It is certainly possible that other topics related to search engines have been studied but were not included in this framework’s underlying sample of 52 articles.

3. METHODOLOGY

In order to understand which academic area exhibits most of the research efforts for search engine related topics, journals in the fields of entrepreneurship, marketing, and general business management were examined. Other research fields such as computer science and information system will most likely have done research on the highly technical subject of online search engines. However, as already argued by Lamberton and Stephen (2016), these academic fields are beyond the scope of this literature review’s objective, which is primarily business-related.

Journals of the two best rating categories in the VHB-JOURQUAL 3 ranking were included in this preliminary search. VHB-JOURQUAL is a rating list of academic business journals compiled by Verband der Hochschullehrer für Betriebswirtschaft (VHB), a German organization of business studies professors.

VHB rates journals with letters from A+ to D. Their final ratings are derived from the median of all ratings submitted by VHB’s more than 1,100 academic members. An A+ rating signifies an outstanding, world-leading academic business journal. Journals given an A-rating belong to the leading ones within academic business disciplines. B-rated journals are important and reputable academic publications. Journals with a C-rating are recognized academic business publications; a D-rating signifies a general journal in the academic business area without special recognition.

As a temporal limitation, only publications of the past twenty years were considered. The intention was to make sure that all years since the foundation of today’s leading search engine Google in 1998 were covered. It would have been particularly interesting to compare the research in this field before and after the market entry of this disrupting company. However, as in Lamberton and Stephen (2016) or Schibrowsky et al. (2007), the time restriction had to be made due to the scope of work. In order to cover the full twenty years, all articles published from 1997 to 2017 were searched and in addition the year 2018 until the month of July.

The keywords that were used to filter the database search results were an important tool to efficiently identify related publications. The selected keywords were quite generic search-engine-terms and could appear in the title, the abstract, or a paper’s keyword section: Search Engine, Paid Search, Sponsored Search, Google, Yahoo, Bing, SEO, Search Advertising, Search Marketing. This enabled a very broad search that included every paper that was remotely connected to a search-engine-related topic.

The entrepreneurship rating list showed four A-rated and eight B-rated journals (VHB, 2015b). Out of these twelve top-rated journals in the field of entrepreneurship, only five journals, two A-rated and three B-rated journals, published a total of eight articles that matched the search query for the defined time frame. Interestingly, when the four VHB-top-rated journals for entrepreneurship Research Policy , Journal of Business Venturing , Entrepreneurship Theory and Practice , and Strategic Entrepreneurship Journal were searched, only five matching articles could be found. Furthermore, after conducting a preliminary analysis of each article’s abstract, none of the matching articles were relevant for the search engine literature review.

The VHB rating list for general business management featured eight A+-rated and nine A-rated journals (VHB, 2015a). Of that list, five A+-rated journals published thirty articles that matched the search query, whereas four A-rated journals published four potentially relevant articles. The abstract analysis showed that sixteen of the thirty-four articles were relevant for a literature review.

The VHB rating for marketing journals showed a total of four A+-rated and seven A-rated journals (VHB, 2015c). The four A+-rated journals published sixty research articles that matched the search query in the time frame while the A-rated journals published eleven. The first abstract analysis resulted in forty-three relevant articles from A+-rated journals and seven relevant articles from A-rated journals.

This confirms the finding of the last significant digital marketing literature review by Lamberton and Stephen (2016) that digital marketing is predominantly researched in the academic discipline of marketing rather than in other business disciplines.

After this initial scan of reputable business journals, the final database search query was decided, and the results were summarized and categorized, which will be explained in the following.

3.1. Database Search Query

In order to limit the search for academic journals to the ones that were used by Lamberton and Stephen (2016), only articles from Journal of Marketing Research , Management Science , Journal of Marketing , Journal of Consumer Research, and Marketing Science were taken into consideration. These are the journals that were given an A+-rating in the VHB-JOURQUAL 3 marketing rating and the A+-rated journal Management Science from the general business management rating list. This makes this literature review to a certain extent comparable with the literature review by Lamberton and Stephen (2016), who analyzed the exact same set of journals. As mentioned before, other journals with lower ratings in the VHB-JOURQUAL 3 list also published articles with relevancy to search engine topics, but the scope of this literature review had to be limited for practical reasons.

The database search query included the aforementioned topic-related keywords, the five selected A+-rated journals, and the time frame of more than 20 years from 1997 to mid 2018 (July). The database of Business Source Premier, as in Schibrowsky et al. (2007), was accessed through the Web of Science interface 1 to search and filter relevant papers. The search resulted in 78 articles that matched the query. The abstract analysis resulted in 26 articles that were not relevant, leaving 52 articles that researched search-engine-related topics.

The irrelevant articles were mostly mismatched, because they used search engines as a part of their studies’ methodology. Oftentimes, search engines are used in field experiments or historical search data is analyzed to study an unrelated topic. Sometimes, articles mismatched because the abstract contained one of the brand names, e.g. Yahoo, Google, or Bing, but just referred to them as an example and did not actually research a topic related to search engines.

3.2. Summary and Categorization

In order to determine the primary topic area of each article, the content of each article was analyzed. The analysis showed that 26 articles were not relevant for this literature review, e.g. because the research merely used search engines as a tool but not as their main research object. Information about each study’s sample, research method, and key findings were summarized for each of the remaining 52 articles, subsequently assigning them to relevant research topic categories (see the literature summary table in APPENDIX). At first, detailed research topic descriptions were given to each article. Later, these detailed topic descriptions were merged into broader topic categories. The topic categories were necessary to create the overarching conceptual research framework that structures all research connected to search engines. The framework aids in categorizing existing and future research as well as in uncovering blind spots, which could be potential future research areas. The literature table also contains the scientific method that was used, classified into quantitative as well as qualitative research or economic model as research method. The economic model was specified as quantitative when the study used empirical data to test the model, and as qualitative if only theoretical economic modelling was applied.

4. EMPIRICAL FINDINGS

4.1. analysis and classification of research articles.

Before the content analysis of each article was carried out it was helpful to create a diagram that showed the number of articles published per year (Figure 2). This made it possible to understand in which time periods of the late 20th and early 21st century most of the research was conducted. Figure 2 reveals that most articles have been published since 2008. Only three articles were published in the ten years from 1997 to 2007. The year with most publications was 2011 with eleven articles, followed by 2014 and 2015 with nine articles each.

The analysis of all 52 academic research articles showed that almost all studies touched multiple different topics connected to search engines. Figure 3 illustrates the popularity of certain topics and exposes the topics that were only touched a few times over 20 years.

Research articles that only dealt with a single topic were about search engine usefulness, organic page rank, paid search effectiveness, bidding strategies, and search engine utilization (Bradlow & Schmittlein, 2000; Danaher & Dagger, 2013; Du, Hu, & Damangir, 2015; Katona & Sarvary, 2008; Rutz, Sonnier, & Trusov, 2017; Skiera & Abou Nabout, 2013).

The most frequently researched topics were paid search effectiveness (23 articles), search engine profitability (15 articles), ad position performance (14 articles), and search engine utilization (14 articles). Topics that were investigated in the least amount of articles were attribution models (H. (Alice) Li, Kannan, Viswanathan, & Pani, 2016; H. Li & Kannan, 2014), search engine usefulness (Bradlow & Schmittlein, 2000; Chesnes, Dai, & Zhe Jin, 2017) and click fraud (Wilbur & Zhu, 2009).

4.2. Search Engines

4.2.1. search engine usefulness, utilization, and user experience.

Before search engines became an essential part of everyone’s online navigation, early studies in the field were concerned with the usefulness of search engines. Bradlow and Schmittlein (2000) searched 20 marketing phrases with several popular search engines in order to evaluate how well they covered the world wide web. This was at an early stage of search engine development and today’s market leader Google was not even included in the study. Bradlow and Schmittlein (2000) found that search phrase and URL characteristics significantly affect search engine outcomes. They already understood that the total number of web pages indexed by a search engine is the biggest factor for search effectiveness. A common finding across engines was that the more links were included on a web page, the more likely that web page would be found. Because the indexing was not extensive, users needed to use multiple engines to achieve good results.

The only other study in the sample that was concerned with search engine usefulness researched the effect of prohibiting foreign pharmacies to buy sponsored search ads (Chesnes et al., 2017). Certain foreign pharmacies that were not certified by the National Association of Boards of Pharmacy (NABP) were banned from paid search ads. Chesnes et al. (2017) found that banned pharmacies without any other certification experienced much less clicks than before the ban. On the search results page for a certain drug, the search engines displayed general drug information, such as side effects, usage advice or warnings, instead of the sponsored links. This increased usefulness of search engine results for consumers could also be seen in the decreased link clicks on organic listing of banned pharmacies.

At the beginning of the rise of search engines, it was also studied how users utilize the search services to navigate the internet. Telang, Boatwright and Mukhopadhyay (2004) studied the internet-use behavior from 1998 to 1999 and found that users tend to decrease their search engine usage over time. They also achieved a precise prediction of visit timing with a mixture model that explicitly incorporates user schedules. Nonsearch features like email or news tend to increase the amount of search engine visits and prevent users from reducing their search engine use, leading to a regular and continued use.

Concerning consumers’ utilization of sponsored ads, Yao and Mela (2011) found that around 10% of search engine users are responsible for 90% of the clicks on sponsored search results. The consumers that click the most also give greater value to the ad position, i.e. top ad positions are clicked more often by people who click a lot of ads. Once the consumer has clicked the ad and found the website that is useful for them, they generally return by typing in the website address directly (Rutz, Trusov, & Bucklin, 2011). The notion that two different user groups for search engines exist, high-involvement and low-involvement consumers, was also supported by Jerath, Ma and Park (2014) and Chan and Park (2015). According to Jerath et al. (2014), high-involvement consumers typically search for less popular keywords and show more (sponsored) clicks per search compared to low-involvement consumers, who search for popular keywords. Edelman and Lai (2016) on the other hand divide search engine users into searchers who consider conspicuous search results (salience searchers) and searchers who assess a search result’s relevance (relevance searchers).

Particularly interesting is Hu, Du, and Damangir's (2014) finding that the interests reflected in Google users’ search queries may be considered as representative of the general population. In connection with Joo, Wilbur, Cowgill and Zhu (2014), it shows that search engine utilization can reflect society’s behavioral patterns. Joo et al. (2014) found that advertisements on television increase the advertised brand’s share of keywords searched and also increases the number of searches in the relevant product category. Hence, it is not surprising that product feature search trends may be treated as reflective indicators of feature importance trends (Du et al., 2015). These insights have managerial implications that go beyond sales forecasting but can also be used in product design, budget allocation, advertising design and budgets, and product and inventory planning (Du et al., 2015).

The user experience seems to have a great impact on search engine utilization and also on search engine profitability. As mentioned before, search engines use additional services and features, e.g. email services, news services, or file storage, as means for user retention (Telang et al., 2004). At the same time, removing filtering and sorting features of the search engine would decrease its usability and would cause a 2.9% reduction of search engine revenue (Yao & Mela, 2011). User experience is therefore directly related to search engine profitability.

Ironically, providing a good user experience can also hurt search engines’ revenues: High-quality websites that are ranked higher organically provide a great user experience, as it simplifies the search for the best result. This attracts more consumers to the search engine but hurts its revenues because the high-quality websites buy fewer paid search results (Berman & Katona, 2013). Nevertheless, when using the customer-utility-based ranking model to rank paid search ads, the highest search engine revenue occurs (Ghose, Ipeirotis, & Li, 2014).

4.2.2. Search Engine Profitability

Chen, Liu and Whinston (2009) researched the fundamental question of search engine advertisement, which is the share structure problem, i.e. how many advertising resources to give to the highest and lowest bidder. According to the model, the share structure problem can be solved in dependency on advertisers’ price elasticity, return factors of different shares, and changes in total advertising resources. In today’s search engine advertising programs, these parameters are called daily budget, click-through-rate (CTR), and the available space on different devices, such as smartphones, tablets, or desktop computers.

1 Website Address: http://www.webofknowledge.com

• No comments yet.

Similar texts

Search Engine Advertising (SEA). Goog...

Konzept zur Anwendung von Web Intelli...

Erfolgsfaktoren des Search Engine Mar...

Search Engine Marketing for the Chine...

International Search Engine Marketing...

Search Engine Marketing. Chancen und ...

"Search Engine Advertising"...

Search Engine Optimization. Eine ethi...

Keyword Relevance in Search Engine Op...

Search Engines. About searching &...

Online Marketing

Search Engine Advertising. Wie gestal...

Marketing-Automation. Evaluierung ein...

Online-Marketing mit niedrigem Budget...

A study of Marketing and Online Marke...

Social-Media-Marketing-Maßnahmen. Bed...

Das Online-Marketing der Bergzeit Gmb...

Leveraging digital marketing for growth

The Effects of Subliminal Cues on Inf...

Upload papers

Your term paper / thesis:

- Publication as eBook and book - High royalties for the sales - Completely free - with ISBN - It only takes five minutes - Every paper finds readers

Publish now - it's free