Summary
Research gets transmitted in many forms — demos, code,
talks, and more — but its formal report most often occurs
through a written paper. The paper explains the problem,
the approach, and an evaluation. When ready, the academic
submits the paper for review by other academics; once it
passes this peer review process, the paper appears at a
venue such as conference or journal. Since most research is
disseminated in the form of papers, it's critical to be
able to read research papers and make sense of them.
Your first assignment will be to read a paper in the
area of your project interest, and to synthesize its main
argument into a detailed outline. In addition, you will be
getting set up with any prerequisites for your section.
Don't underestimate the time required for either of
these!
Part 1: Read a Paper
Read a paper and outline its argument and structure.
Each section has listed one paper below. Find yourself a
quiet place and work your way through it. Don't worry if
you can't understand every detail; focus on understanding
the paper's big ideas and how they are argued. It can take
time to read a paper — don't feel discouraged if it takes
you a long time.
Outlining a Paper
Your outline should recompress the paper back into an
outline format that explains the paper's argument. We
suggest the following structure of one paragraph per
bullet, with an example outline following the
description:
- Title: What paper did you read?
- Problem: What problem is it solving? Why does this problem matter?
- Assumption in prior work: What was the assumption that prior research made when solving this problem? Why was that assumption inadequate?
- Insight: What is the novel idea that this paper introduces, breaking from that prior assumption?
- Technical overview: How did the paper implement that insight? I.e., What did they build, or what did they prove, and how?
- Proof: How did the paper evaluate or prove that its insight is correct, and is better than holding on to the old assumption?
- Impact: What are the implications of this paper? How will it change how we think about the problem?
You may adapt this structure if needed for the paper you
are outlining; check with your TA first if your proposed
structure deviates by more than one bullet.
Example, for the paper Flash
Organizations: Crowdsourcing Complex Work by Structuring
Crowds As Organizations:
- Title: Flash Organizations:
Crowdsourcing Complex Work by Structuring Crowds As
Organizations
- Problem: Crowdsourcing has been
used successfully for many goals that can be decomposed
into small, modular microtasks, but it has struggled to
achieve more complex goals such as design and
engineering. For example, tasks such as image labeling
work because it's modularizable, but design is
interdependent and requires adapting as you go, so
crowdsourcing has succeeded at image labeling but
failed at design. If crowdsourcing is limited to
modular tasks, then it will never be able to achieve
goals of meaningful complexity, which will limit its
impact on the world.
- Assumption in prior work: Prior
work all takes an algorithmic model of crowdsourcing:
the programmer specifies who does what, and when, in a
kind of big algorithmic recipe to follow. This
assumption shows up in goals ranging from Wikipedia
("edit this page"), to interactive crowd-powered
interfaces ("find errors" --> "fix errors" -->
"verify fixes"), to open source software ("create a
module with this fixed API").
- Insight: This paper proposed that
instead of coordinating crowds as we do algorithms,
that we should be coordinating crowds as we do with
organizations. They propose a series of
computationally-enhanced versions of the structures
that organizations use — roles, tasks, hierarchy, and
so on — and introduce the idea of a flash organization,
which is a rapidly assembled collective of online
collaborators who use these computational
organizational structures to coordinate.
- Technical overview: The authors
created a system called Foundry that implements these
ideas. Foundry is a web interface that connects to the
Upwork online labor marketplace to draw on-demand
expertise. It uses a combination of
first-come-first-served hiring queues and Slack
integration to bring workers onboard and keep them
updated. It introduces an adaptation model drawn from
the metaphor of code branching and merging to enable
the organization to adapt.
- Proof: Three non-crowdsourcing
experts used the system to convene and lead flash
organizations to achieve proof-of-concept complex
goals. These experts created (1) a tablet system for
EMTs to report medical trauma cases enroute to the
hospital, (2) a card game for storytelling, which was
playtested and iterated upon, and (3) an
enterprise-grade event planning system that had to meet
branding and security standards.
- Impact: Flash organizations offer
a broad new view of crowdsourcing — one that's not
rooted in Tayloristic algorithms, but instead in an
organizational metaphor. This approach can achieve far
more complex outcomes, enabling crowdsourcing to apply
to a broad new class of problems. It has implications
for the future of work (how do we protect labor
rights?), for organizations (what will organizations
look like in the future if flash organizations are
widely deployed?), and for collaboration (will we all
work remotely?).
Papers
Here are the papers per section. Look at your section's
project list and pick the paper that corresponds to a
project that intrigues you.
Note: If you encounter a "Get Access" paywall
on the below links (e.g., for IEEE or ACM), make sure you
are using Stanford's EZproxy.
Kanishk's section:
Brando's section — pick one of:
Kexin's section:
Part 2: Section Starter Task
Complete starter task for your section
Kanishk's section: starter task details.
Brando's section: starter task details.
Kexin's section: starter task details
Submission
You are expected to submit two PDFs and one form for this assignment:
- Submit a PDF with your paper outline on Canvas.
- Submit a PDF for your section’s starter task on Canvas.
- Submit your responses to the project ranking form linked above.
Grading
Your outline will be graded on the following rubric:
- Accuracy: does the description correctly describe the paper?
- Completeness: does the description capture all the important ideas in the paper? (Not all the ideas! All the important ideas.)
- Clarity: does the description convey the ideas understandably to the reader?
In addition, your starter task will be graded based on completion.