Perspectives on Data Science for Software Engineering by Thomas Zimmermann & Laurie Williams & Tim Menzies

Author:Thomas Zimmermann & Laurie Williams & Tim Menzies [Thomas Zimmermann]
Language: eng
Format: epub
Tags: Core Programming, Engineering
Publisher: Morgan Kaufmann
Published: 2016-07-13T21:00:00+00:00

How Far Can We Use the Outcomes From Systematic Review to Make Decisions?

So, if your doctor can make decisions about how best to treat you using the outcomes of a systematic review, can we make our technical decisions in the same way? In principle, the answer is yes, although in practice it is rather less clear-cut for software engineering. This is largely because the form of a clinical study usually involves making comparisons between the effects of the treatment and some placebo, and with the participants being recipients of the treatment, making it possible to use statistical meta-analysis to synthesize the outcomes. In contrast, computing studies usually have a range of forms, and the participants are often asked to perform skilled tasks within organizational contexts that may introduce many other “confounding factors,” and so complicate eventual synthesis.

In a recent study we examined 216 systematic reviews, published between 2004, when the use of systematic reviews began to be adopted in software engineering, and the end of 2014, to see if they contained material that might be used to support the teaching of core software engineering ideas as well as to provide advice about practice. From these, we identified 59 studies that were considered to provide useful experience about such topics as cost modeling, requirements elicitation, model-driven engineering, agile methods, the needs of start-up companies, etc. (see the Appendix in [2]).

In computing, it is unusual for the adoption of a new or different technical practice (eg, pair programming) to make a large difference when compared with the effect of using another. So none of the studies had definitive results showing that one technique was always better than another, although a number embodied useful experiences about the situations in which it might be appropriate to employ a particular technique.

We should not be too surprised at this. Back in 1987, Fred Brooks Jr. explained why the nature of software made it unlikely that the “silver bullet” solutions desired by managers could ever be feasible for software development activities [4]. So, when it comes to making decisions in computing, consulting the outcomes of a systematic review is unlikely to make the decision for you, but it is likely to provide you with some important insight into what does work, when it is most likely to work, and possibly why, depending on the degree of knowledge translation provided. And knowing what factors may be important can still usefully make it possible to come to an evidence-informed decision. (Given that every patient the doctor sees has their own set of personal factors, clinical decisions are usually evidence-informed too.)

Will things change in the future? Well, this is still early days for secondary studies, with experience from only little more than a decade of performing systematic reviews in software engineering, much of which has been a learning process. With this growing experience, it is likely that more and better systematic reviews will be performed. We may also expect better primary studies, performed to meet needs identified by systematic reviews.

So,

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.