Evaluation Process


Journal evaluation and selection is ongoing at ESSCI with journals added to and deleted from the database as frequently as every few weeks. Each year, the ESSCI editorial staff reviews a certain amountjournals and selects around 20% of the journals evaluated for inclusion in the ESSCI database. Moreover, existing journal coverage in ESSCI products is constantly under review. Journals now covered are monitored to ensure that they are maintaining high standards and a clear relevance to the archives in which they are covered. The journal selection process described here is applied to all journals in ESSCI. Some special consideration is given in the evaluation of social science and arts and humanities journals, particularly with regard to citation analysis.

 

Many factors are taken into account when evaluating journals for coverage, ranging from the qualitative to the quantitative. The journal's basic publishing standards, its content, the international diversity of its authorship, and the citation data associated with it are all considered. No one factor is considered in isolation, but by combine and interrelate the data, the editor is able to determine the journal's overall strengths and weaknesses.

 

ESSCI editors who perform journal evaluations have educational backgrounds relevant to their areas of responsibility. Because they monitor virtually every new scholarly journal published, they are also experts in the literature of their fields.

 

ESSCI also notes if the journal follows international editorial conventions, which optimize irretrievability of source journals. These conventions include informative journal titles, fully descriptive journal titles and abstracts, complete bibliographic information for all cited references, and full address information for every author.

 

English is the universal language of science at this time in history. It is for this reason that ESSCI focuses on journals that publish full text in English or at very least, the bibliographic information in English. There are many journals that publish only bibliographic information in English with full text in another language. However, going forward, it is clear that the journals most important to the international research community will publish full text in English. This is especially true in the natural sciences. In addition, all journals must have cited references inthe Roman alphabet.

 

Application of the peer-review process is another indication of journal standards and signifies overall quality of the research presented and the completeness of cited references.It is also recommended that whenever possible, each journal is published with information on the funding source supporting the research presented.

 

Content

 

As mentioned above, an essential core of scientific literature forms the basis for all scholarly disciplines. However, this core is not static — scientific research continues to give rise to specialized fields of studies, and new journals emerge as published research on new topics achieves critical mass. ESSCI editors determine if the content of anjournal under evaluation will enrich the database or if the topic is already adequately addressed in existing coverage.
With an enormous amount of citation data readily available to them, and their daily observation of virtually every new scholarly journal published, ESSCI editors are well positioned to spot emerging topics and active fields in the literature.

 

International Diversity

 

ESSCI editors look for international diversity among the journal’s contributing authors, editors, and editorial advisory board members. This is particularly important in journals targeting an international audience. Today’s scientific research takes place in a global context and an internationally diverse journal is more likely to have importance in the international community of researchers.

 

As the global distribution expands into virtually every region on earth, the importance of regional scholarship to our emerging regional user community also grows. Selection criteria for regional journals are the same as for international journals although citation analyses play a somewhat different role in the outcome. For example, the importance of the regional journal would be measured more in terms of the specificity of its content. It will enrich our coverage of a particular subject or provide studies with a specific regional perspective.

 

All regional journals selected must have English-language bibliographic information (title, abstract, keywords), and be peer reviewed. Cited references must be in the Roman alphabet.

 

Citation Analysis

 

The ESSCI evaluation process is unique in that our editors have a wealth of citation data available to them. The importance of interpreting and understanding these data correctly cannot be emphasized too strongly. Using quantitative citation data to measure impact is meaningful only in the context of journals in the same general discipline. In some areas, particularly in the arts and humanities, it may take a relatively long time for anjournal to attract a meaningful number of citations. But in other areas, such as the life sciences, it is not unusual for citations to accrue rapidly and peak after two or three years. The facts must be taken into consideration if citation data are to be used correctly.

 

Citation analysis takes place on at least two levels. We look for citations to the journal itself, as expressed by impact factor and/or total citations received. We also examine the citation record of the contributing authors, which is a useful study in evaluating new journals where a citation history at the journal level does not yet exist.

 

Likewise, established journals that are not covered are often re-evaluated. These journals can experience new growth in citation impact resulting from changes such as translation into English, change in editorial focus, medium, etc.

 

Self-citation rates are also taken into consideration. The self-cited rate relates anjournal’s self-citations to the number of times it is cited by all journals, including itself. A high volume of self-citation is not unusual or unwarranted in journals that are leaders in a field because of the consistently high quality of the papers they publish and/or because of the uniqueness or novelty of the subject matter. Ideally, authors reference the prior publications that are most relevant to their current results, independently of the source journal in which the work was published. However, there are journals where the observed rate of self-citation is a dominant influence in the total level of citation. For these journals, self-citation has the potential to distort the true role of the title as a participant in the literature of its subject. If we determine that self-citations are being used improperly, the journal’s impact factor will not be published and the journal may be considered for deselect from the Web.

 

All social science journals undergo the same thorough evaluation as journals in the natural sciences. Standards, content, international diversity, and citation data are all considered. Standard citation metrics are analyzed while keeping in mind that overall citation rates in the social sciences are generally lower than those in the natural sciences.

 

Regional studies have special importance in the social sciences as topics of local, rather than global, interest are often the subject of scholarly research

 

Publishing standards, including timeliness, are important in the evaluation of arts and humanities journals. Citations patterns in the arts and humanities, however, do not necessarily follow the same predictable pattern as citations to social sciences and natural sciences journals. In addition, arts and humanities journaljournals frequently reference nonjournal sources (e.g., books, musical compositions, works of art, and literature).

 

Electronic Journals

 

Publishing standards, editorial content, international diversity, and citation analysis are all considered when evaluating a purely electronic journal.

 

Evaluating the timeliness of publication of an e-journal may require a somewhat different approach. If the e-journal is publishing distinct issues, the editor looks for the appearance of these in a timely manner. However, when an e-journal publishes articles one at a time rather than collecting articles for release as an issue, the editor simply looks for a steady flow of articles over several months.

 

The format of electronic journals is extremely important. Following is a set of guidelines for electronic journal formats. Following these guidelines helps ensure correct citation of articles and reduces the possibility of ambiguity in citation of articles.

 

Ensure that it is easy to identify the following elements:

 

Journal title
Year of publication
Volume and/or issue number (if applicable)
Article title
Page number or article number (one or the other is required), if your journal has page numbers and article numbers, list them separately and not merged together.
Authors names and addresses
Label all article identifiers such as DOIs, PIIs, and article numbers
A complete table of contents for each issue that includes the page/article number for each article (unless journal is being published as single articles)
Labeling these identifiers in both source articles and in citations helps ensure their proper use by those referencing the article and correct labeling by abstracting and indexing firms.

 

Each article must be assigned a unique page number or article number (whichever numbering scheme is being used) within any one given issue. Additionally, article numbers must be unique within an entire volume number. If the same article numbers are repeated in each issue within a volume, ambiguities will result when citing the original article. A citation to V20, art. 1, May 2002 (even when adding the author's name) would be difficult to find if V 20, art. 1 June 2002 also exists. Avoid duplication of article numbers.

 

References to your electronic journal - instruct authors to include the following information when citing your e-journal:

 

Journal title (uses one standard abbreviation for your journal; avoid acronyms that may be confused with other titles)
Volume number (if applicable)
Issue number (if applicable; within parenthesis)
Page number and/or article number (clearly identifying the article number as such)
Year of publication