Research Track

  • Who: Teams of 2-3
  • Objectives: Research

Overview

In lieu of all other grading categories, a successful attempt at the Research Track will entirely determine your grade in the class. Should you choose to accept the Research Track challenge, that means you do not have to complete any other graded assignments.

The Research Track involves writing a publication-quality paper. Successful projects will be submitted as a short paper to the visualization conference IEEE VIS (deadline April 30). You learn so much in the process of research, and this can be a really valuable experience if your group is up for it. This must take place with your project group. So if this is something you are interested to pursue this semester, do consider choosing your project group accordingly.

If you want to pursue the Research Track, talk to the instructor and TA early and often! When in doubt, ask questions.

Why Research Track?

I am a big advocate for the value of research -- even failed research! The process can be transformative to the way you think about and approach problems.

This is risky! I do not recommend the Research Track for everyone! Who should consider this route?

  • Graduate students
  • Undergraduate students who want to attend graduate school
  • Students who want to publish
  • Students who want to explore visualization research
  • Students who are willing to put in hard work
  • Students who are comfortable to self-manage their time and set internal milestones

Expectations

Your deliverable at the end of the semester will be a paper that is ready to submit to IEEE VIS. Pay attention to the required deadlines for paper submissions. You are responsible for ensuring that you follow submission requirements and meet the deadline.

  • Consider paper types -- what is the type of contribution you will make? A system? An empirical study? A theoretical framework? Check out this helpful blog post on What is Visualization Research?
  • Look at prior publications from EuroVis and IEEE VIS to get a sense of typical paper structure.
  • Your paper must satisfy the formatting requirements (max 4 pages of content + 1 page of references in VGTC Conference Style).

That being said, you are not on your own! The TA and I want to help you maximize your chances of success.

  • Before you choose the Research Track, you must indicate your intention to the instructor and the TA by the end of Add/Drop/Swap at the latest.
  • You must schedule check ins with the instructor and TA on a biweekly basis, so we can provide feedback and guidance. Note: each biweekly update should consist of meeting slides to communicate your progress -- consider dividing it up into (1) what you have been working on, (2) what you are stuck on or struggling with, and (3) what you need help with.
  • We will make ourselves available to discuss project ideas, research value, project pacing, paper outlines, provide feedback on written drafts, etc. Write early and often. The more you take us up on the opportunity for feedback, the better off you'll be!

If you are not confident you will be able to put in the necessary work, you will likely want to hedge your bet and ensure your grade will not suffer by working on the other assigned coursework. Your findings need not necessarily be groundbreaking, but rushed efforts that result in 4 pages of rambly writing will not constitute a successful attempt at the Research Track!

Research Overview

To write a research paper, you first need to have a reasonable sense of what constitutes research! Below are some steps you will want to think through. Your paper will need to address these in some fashion.

Define a problem: What is a problem? Whose problem is it? (researchers, practitioners, industry, government, everyone?)

Come up with a solution: What are existing solutions? How well do they work? Do we need a new solution? Why?

Demonstrate effectiveness: Evaluation involves demonstrating that the solution works on the problem. How to evaluation depends on the problem statement. If the problem is that an algorithm is slow, demonstrate that your solution shows quantitative improvement. If the problem is that a domain user cannot do X effectively, show (in a qualitative study) that your solution helps the user do X easily. One option to consider for evaluation is “A nested model for visualization design and validation”, Tamara Munzner, TVCG, 2009.

So what? Perhaps the hardest part of doing research is to answer the question of “so what.” Connected to the “identify problem” consideration. In some cases, the “so what” is self-evident. For example, if you built a tool to help a domain user do X more effectively, your “so what” depends on: How many of these domain users are there? How important is X (and doing X well)? How often do they use your tool? But other times, the “so what” is less obvious… Consider “The Value of Visualization”, Jarke van Wijk. IEEE VIS, 2005.

  • Suppose you developed a tool to help one doctor visualize gene data: (i) this doctor ended up finding a cure for cancer using your tool, (ii) this doctor used your tool and wrote a paper based on the result, (iii) this doctor used your tool to generate an image for a paper
  • Suppose you wrote a library to make programming visualizations easier: (i) most visualizations in the world are built using your library (e.g. d3), (ii) you used your library to start a company (e.g. plotly), (iii) you wrote a paper, but adoption never took off (e.g. infovis toolkit)
The point is, the “so what” isn’t always obvious.

Writing a paper: you need to make it clear to the reviewer “why” your work is relevant. Practical Impact: for your own purpose, before you embark on a research project, understand why you are doing what you’re doing.

Resources

Visualization Conferences:

Visualization Research:

How to Read a Paper:

How to Review a Paper:

How to Write a Paper:

How to Give a Presentation:

Resources courtesy of Topics in Data Visualization at Harvard from Johanna Beyer

Grading

Successful completion of the Research Track means that your grade will entirely depend on the quality of the paper that you produce in the end of the semester, regardless of your performance (or lack thereof) on other assignments. Work that is incomplete (e.g., the system you built doesn't work, or the evaluation was not completed) or clearly thrown together (e.g., the paper is unintelligible and lacks any semblance of consultation with the instructor or TA) will not be graded lightly. Your research need not be successful per se (e.g., you may fail to find a sigfnificant difference in your evaluation), but it should be the result of concerted efforts.

Research will be graded based on:

  • Problem definition: You should have a compelling motivation for your problem. This will likely appear in the Introduction section of your paper.
  • Related Work: Your work should be contextualized in recent relevant research.
  • Methodology: Your research should follow a rigorous and well-justified methodology. This is the part you will likely need the most help from the TA and insutrctor on.
  • Evaluation: Your research should be evaluated. How can you demonstrate that your theory or what you built is correct?
  • So-What: Now that your work is completed, how is the field affected? What can we do newly or better that we couldn't do before?
  • Novelty: Does your research contribute a novel finding or novel perspective?
  • Writing quality: Is your paper easy-to-read and follow?