Haystack Group

Research on Information Access, Analysis, Management, and Distribution

Massachusetts Institute of Technology
Computer Science & Artificial Intelligence Laboratory
32 Vassar St., Cambridge, MA 02139

Our goal is to make it easier for people to collect, organize, find, visualize, and share their information. We are an interdisciplinary group of researchers blending approaches from human-computer interaction, social computing, databases, web infrastructure, information retrieval, artificial intelligence and the semantic web.

Recent News

Thoughts from the Haystack Blog

» full blog and news

Latest Publications

» complete list of publications

Active Projects

» complete list of active and inactive projects

TwitInfo: Aggregating and Visualizing Microblogs for Event Exploration. CHI, 2011.

Adam Marcus, Michael Bernstein, Osama Badar, David Karger, Samuel Madden, and Rob Miller

Microblogs are a tremendous repository of user-generated content about world events. However, for people trying to understand events by querying services like Twitter, a chronological log of posts makes it very dif�cult to get a detailed understanding of an event. In this paper, we present TwitInfo, a system for visualizing and summarizing events on Twitter. TwitInfo allows users to browse a large collection of tweets using a timeline-based display that highlights peaks of high tweet activity. A novel streaming algorithm automatically discovers these peaks and labels them meaningfully using text from the tweets. Users can drill down to subevents, and explore further via geolocation, sentiment, and popular URLs. We contribute a recall-normalized aggregate sentiment visualization to produce more honest sentiment overviews. An evaluation of the system revealed that users were able to reconstruct meaningful summaries of events in a small amount of time. An interview with a Pulitzer Prize-winning journalist suggested that the system would be especially useful for understanding a long-running event and for identifying eyewitnesses. Quantitatively, our system can identify 80-100% of manually labeled peaks, facilitating a relatively complete view of each event studied.

The Trouble with Social Computing Systems Resaerch. alt.chi, 2011.

Michael Bernstein, Mark Ackerman, Ed H. Chi, and Rob Miller

Social computing has led to an explosion of research in understanding users, and it has the potential to similarly revolutionize systems research. However, the number of papers designing and building new sociotechnical systems has not kept pace. We analyze challenges facing social computing systems research, ranging from misaligned methodological incentives, evaluation expectations, double standards, and relevance compared to industry. We suggest improvements for the community to consider so that we can chart the future of our field.

The Schema-Independent Database UI: A Proposed Holy Grail and Some Suggestions. CIDR, 2011.

Eirik Bakke and Edward Benson

If you have ever encountered a piece of highly domain-specific business software, you may have noticed that it was largely a graphical front-end to some relational database. You may also, in fact, have avoided using the system at all--studies show that information workers prefer to dump their data into spreadsheets, a general and more familiar tool which, unfortunately, is poorly suited for many standard database tasks. It is time that we stop streamlining the process of creating a new application for every schema, and that we instead develop the visual query languages that will let end-users access the full power of relational database management systems from a simple and unified interface. Once information workers can create, manage, and query real databases with the same ease as they routinely manipulate spreadsheets today, they will never return to their schema-dependent, consultant-made, and oddly-colored Microsoft Access applications.

A Spreadsheet-Based User Interface for Managing Plural Relationships in Structured Data. CHI, 2011.

Eirik Bakke, David R. Karger, and Robert C. Miller

A key feature of relational database applications is managing "plural" relationships--one-to-many and many-to-many--between entities. However, since it is often infeasible to adopt or develop a new database application for any given schema at hand, information workers instead turn to spreadsheets, which lend themselves poorly to schemas requiring multiple related entity sets. In this paper, we propose to reduce the cost-usability gap between spreadsheets and tailor-made relational database applications by extending the spreadsheet paradigm to let the user establish relationships between rows in related worksheets as well as view and navigate the hierarchical cell structure that arises as a result. We present Related Worksheets, a spreadsheet-like prototype application, and evaluate it with a screencast-based user study on 36 Mechanical Turk workers. First-time users of our software were able to solve lookup-type query tasks with the same or higher accuracy as subjects using Microsoft Excel, in one case 40% faster on average.

Soylent: A Word Processor with a Crowd Inside. UIST, 2010.

Michael Bernstein, Greg Little, Robert C. Miller, Bjoern Hartmann, Mark Ackerman, David Karger, David Crowell, and Katrina Panovich

This paper introduces architectural and interaction patterns for integrating crowdsourced human contributions directly into user interfaces. We focus on writing and editing, com-plex endeavors that span many levels of conceptual and pragmatic activity. Authoring tools offer help with prag-matics, but for higher-level help, writers commonly turn to other people. We thus present Soylent, a word processing interface that enables writers to call on Mechanical Turk workers to shorten, proofread, and otherwise edit parts of their documents on demand. To improve worker quality, we introduce the Find-Fix-Verify crowd programming pat-tern, which splits tasks into a series of generation and re-view stages. Evaluation studies demonstrate the feasibility of crowdsourced editing and investigate questions of relia-bility, cost, wait time, and work time for edits.

source article

Dido: Dido is a web page that contains data and visualizations of it that you can edit right in the page. It's based on our Exhibit framework, but adds the ability to modify what you are looking at.

Exhibit: a lightweight data publishing framework that you can use to create rich mash-ups using just HTML and JSON. No database, no web application whatsoever.

FeedMe: FeedMe makes it easier to send brief e-mails to your friends to share links. Once youve started sharing, it starts recommending friends who might be interested in seeing the post you're looking at. Ideally this makes it even faster to share with more people who would find content relevant, without requiring you to type or switch windows.

Michael Bernstein and Adam Marcus

Jourknow and Information Scraps: What if your computer could do much better than being a digital sticky note application? What if it could remember where you were, or what you meant, and possibly even help you get it done?

Michael Bernstein and Max Van Kleek

NB: a tool for students to ask questions and discuss answers about lecture notes by annotating them.

PLUM: a research platform for investigating how we can make computers more personal while at the same time preserving people's privacy and keeping control within the hands of users.

Soylent: Soylent is a crowd-powered interface: one that embeds workers from Mechanical Turk into Microsoft Word. It uses crowd contributions to perform interactive document shortening, proofreading, and human-language macros. Underlying Soylent is a new programming design pattern called Find-Fix-Verify that splits tasks into a series of generation and review stages to control costs and increase quality.

Michael Bernstein, Rob Miller, David Karger, Bjoern Hartmann, and Mark Ackerman

atomate: Atomate is a programmable web-bot that lets users automate simple tasks through the construction of context-aware reactive behaviors. These behaviors are driven by data from Web (RSS/Atom) feeds, and are specified through a constrained natural language interface.

Max Van Kleek, Brennan Moore, Paul Andre, David Karger, and mc schraefel

list.it: list.it, the Latitudinal Information Scrap Trapper that Indexes Things - is a small, simple note-keeping tool for solving a big, complex task -- helping you manage the tons of little information bits you need to keep track of each day. list.it does this by focusing on speed and simplicity. We have gotten rid of everything except a way to get things in and out quickly, so that you can get things out of your head and somewhere you can access easily any time.

Max Van Kleek, Michael Bernstein, Katrina Panovich, and Greg Varvas