| layout | toc | author_profile |
|---|---|---|
single |
true |
false |
Background: The Software Engineering Research Quality Coalition
(SERQco, pronounced “Sirco”) works based on the perception that many
empirical SE research contributions have (some aspects of) avoidably
low quality.
Objective: SERQco creates evidence of research quality issues and
makes suggestions how to improve.
Method: SERQco operationalizes reasonable lower bounds for some
aspects of research quality and then measures research quality over
many published empirical SE research articles.
SERQco mostly analyzes published research literature, using methods that work with qualitative data but aim at quantitative results. A common approach for simple cases is open coding plus subsequent content analysis. More complicated cases will use individually crafted combinations of methods.
In the normative aspects of its work, SERQco predominantly uses pragmaticist and mildly antipositivist epistemological stances.
SERQco is a loose collective of SE researchers who work together on a project-by-project basis. A project usually aims at doing a single study and producing one or two publications from it.
SERQco works transparently: We identify each analyzed article clearly and publish raw data. This will embarrass some authors, but our research is about works, not people, and is interested in general trends, not in individual cases (which we may still use for illustration).
There are many questions that SERQco might tackle. Some are straightforward to investigate and the results should be acceptable for everybody; others will be trickier or the outcomes more controversial. SERQco considers them all and picks those where the benefit ratio appears favorable.
Examples (in order of increasing difficulty):
- Abstracts that do not summarize
- Conclusions that do not draw conclusions
- Confusing engineering with science: not explaining relevance
- Use of causational language in studies that are merely correlational
- Imprecise method descriptions
- Questionable tacit assumptions
In some countries, “bar” is a legal term referring to the community of
legal practitioners and
connoting the line in a courtroom that separates the spectators
from the participants of a trial;
the “bar exam” is an examination that people with a university legal degree
need to pass to become members of the bar and be admitted to court
or at least to higher courts.
In this spirit, SERQco will demonstrate some conditions that an SE research article should fulfill before it can be considered good research.