Link to pdf

Research Question

Guidelines grid

Section

length
function
goals
structure
checklist
examples

Abstract

50-200 words
Allow readers to judge whether the paper is relevant to them
  • The abstract should give a clear, concise and accurate view of
    • what has been found
    • how it relates to former research
    • why it amounts to state-of-the-art research
  • A general statement introducing the research area of the topic being investigated
  • An explanation of the specific problem addressed (research gap)
  • A review of existing models and theory (if there is any) in relation to the problem being investigated
  • An outline of the proposed model or approach
  • A summary of whether hypotheses/expectations could be confirmed (evaluation) and how this contributes to scientific knowledge
  • There should be keywords under the abstract that summarize where the thesis is about.
  • Avoid citations.

Introduction

2 paragraphs
Position research and pose research question
  • Topic of the research
  • Problem Statement
  • Research Gap
  • Research Question
  • Position the topic of the research in a scientific field, oriented by either:
    • task: Natural Language Processing, Image Classification, Information Retrieval, ...
    • subject: Medical Science, Agricultural Sciences, ...
  • Problem Statement
    • Problem your research sets out to solve
    • Context for value of conducting research in chosen area
    • Why is the problem scientifically relevant?
    • Make trade-offs explicit that play a key role
  • Research Gap
    • Mention key paper(s) for your research
    • How they are related to the research gap?
    • Where is existing knowledge thin?
  • Research Question (refer below)

Methodology

This section can be quite brief. Use most space for your addition. If there is none, the section can be rather brief.
  • Describe what you do
  • Justify it
  • In particular, you pay attention to that new little thing, that change, that great idea, that you add to the existing method.
  • For Information Systems students, it is not always necessary to have a separate methodology section. You can integrate the approach and results in one section, which allows for more efficient usage of the 10 pages. It depends on the kind of research what is best fitting.
  • You have mentioned how others have used the methods you will use in your related work section. In the methodology, you will go into more details about them. This is not a textbook section (readers can find that elsewhere, probably better), nor a place to copy/paste difficult intimidating formulas.
  • A good paper will make clear the type of research design is used, possible by reference to earlier, similar studies.
  • One function of the methodology is to describe what you do. It is not just about what you are using, but how in particular you are using it.
  • The other function is to justify what you do.
    • Why did you choose the current methodology over other plausible alternatives?
    • Does the research design probably measure the effect and is it credible in terms of reliability and validity?
    • How did you go about solving or making progress on the research problem?
    • Did you use simulation or analytic models?
    • What important variables did you control, ignore, or measure?

Methodology: Experimental Setup

This (sub)section can be quite long.
  • The user can replicate your experiments.
  • The user gets a (very) good idea of your used dataset(s).
  • The experimental setup can be either an extensive subsection in the methodology, or it can be made into a separate section. The first approach is traditional but the other is viable as well.
  • Sometimes, especially if you have quite different, experiments/research questions, it makes sense to interleave the experimental setup and the result sections, so the reader does not get lost. It is then helpful to structure clearly in (sub)subsections.
  • If you want or need to tell more, consider using the Appendix or a reference to a nicely structured notebook in which all experiments are done.
  • In the description of the data subsection you paste your most insightful graphs from your EDA, next to the basic statistics on your dataset and descriptions (statistical and/or a population density diagram) of your variables.
  • In the experimental setup (sub)section you give all the settings used in your experiments. All that is needed so that someone with your dataset and your software can replicate your work and obtain more or less (often there are random effects) the same results. Think of:
    • Data pre-processing steps used
    • Hyperparameter settings (for all of them)
    • How you created a train-validate-test split or otherwise did your training and testing
    • Exactly how the used metrics are calculated (think of the difference between micro and macro F1 for classification and how often this is not explicitly stated)•

Results

This section can be quite brief (in words). You will answer your research question in the discussion and conclusion.
The function of the results section is that you give, for each research question, the outcomes of the experiment corresponding to a research question in the form of a table or graphic. In the case of interview-based research, you will show examples and counterexamples by means of quotes.
The reader should be able to use 100% of her brain to understand the outcomes, not to try to figure out what was meant.
  • Structure your section in a way that the reader should only read these two things (and can safely skip all else): the question and the table/figure/quote and the (elaborate) caption.
  • It is important that tables/figures are unambivalent. There is a need for a perfect caption, perfect labels and smart design of table or figure. The reader should be able to use 100% of her brain to understand the outcomes, not to try to figure out what was meant.
  • Specifically, most good computer science papers conclude that something is so many percent faster, cheaper, smaller, significant or otherwise better than something else.
    • Avoid putting the result there in numbers, handwaving results such as "very" or "small".
    • If you must be vague, you are only given license to do so when you can talk about orders of magnitude improvement.
    • There is a tension in that you should not provide numbers that can be easily misinterpreted, but you do not have room for all the caveats either.

Discussion

The function of the discussion section is to compare your results with the state of the art and to reflect upon the results and the limitations of the study.
  • Do not forget to use sub-sections. You can start with a sub-section in which you compare the results of your research with previous studies.
  • Be sure to make use of concrete results.
  • Make use of references. You would like to know whether the found effect size is comparable to previous related studies.
  • Be sure to make use of confidence interval or significance, if applicable, to argue that your results are not just random.
  • If you did not find what was expected, it is important that you go deeper into the possible reasons for this.
    • Are there comparable studies in which no results was found?
    • What would be the most likely scenarios for not finding the results?
    • It is important that you are able to justify that fixable problems with the research set-up are the most likely reason for not finding a result.
    • It will help to find a theory for your results if you use a second dataset with known results to compare with. In that way, you know whether your results are as expected based on the known dataset.
  • The limitations of the study should be noted.
    • Do not make a shortcoming into an excuse. Make it into a strategy for further research.
    • Limitations should be reflected upon by using key concepts like reproducibility, scalability, generalizability, reliability and validity.
    • Make sure that you make clear how you take the limitations into account in the conclusion, if the issues are serious.
  • You can already hint in the discussion at future work to which you come back in the conclusion section.
  • Consider whether there are alternative conclusions consistent with the results presented.
  • What is the value of your research in light of previous research?
  • Does it indeed fill the research gap?
  • This gives a bridge to the conclusion.

Conclusions and Future Work

This section should not be longer than half a page. You will need to be very sharp and concise.
The function of the conclusion section is that you answer all your research question(s).
  • You start with a recap of the scientific relevance and problem statement.
  • You then present the answer to the research questions. You will need to (indirectly) rephrase the questions. For example: “This research aims to answer…”
  • There should be a statement on how the limitations of the study qualify the conclusion.
  • You then go to a statement in which you state what the value of your research.
  • You finish the conclusion section with future work.
  • If the improvement only shows in specific circumstances, it amounts to a qualification.
  • Make sure that your qualification is properly connected to your statements in the discussion.
  • You then go to a statement in which you state what the value of your research has been when compared to the state of the art.
    • What does your research add?
    • What are the implications of your answer?
    • Is it going to change the world (unlikely), be a significant "win", be a nice hack or simply serve as a road sign indicating that this path is a waste of time?
    • Are your results general, potentially generalizable or specific to a particular case?
  • You finish the conclusion section with future work.
  • You can make a link here to the limitations of the study in the discussion section.
  • Be sure that you argue what is the most promising way forward instead of stating which research “must” be done.