The third Identity Function post departs from the Programming Languages community and enters into the world of Human-Computer Interaction. We hear from Matthew Kay, who just finished his Ph.D. at the University of Washington. Matthew Kay’s research deals with both HCI and information visualization. In the fall, he will be starting as an Assistant Professor at the University of Michigan School of Information.
Tell me about yourself.
I grew up in Southern Ontario, Canada, about an hour and a half outside Toronto. Besides Computer Science, I’ve always been interested in art (I have a minor in studio art) and social sciences (particularly psychology and philosophy). These days my interests in art manifest in reading books on design and typography. I usually relax by watching movies, eating cheese, or replaying old video games.
What are you currently working on?
I have been working on building a modified version of OneBusAway [a local bus tracking app] that communicates probabilistic predictions of real-time bus arrival times (instead of just the best estimate). Some of that involves figuring out how to visualize a probability distribution of the time to arrival for some bus in a way that “normal” people can use and understand.
I’ve developed a discrete depiction of continuous probability distributions that was inspired by work on statistical reasoning that suggests people’s probabilistic inferences are better when predictions are presented as frequencies rather than abstract probabilities. That project is now moving towards a deployment and field testing of a version of OneBusAway with probabilistic prediction.
I think everyone in Seattle can relate to pulling up OneBusAway and seeing a bus that is five minutes away, only for it to be four minutes away five minutes later. How does this version convey the information to the user?
We use what I have dubbed “quantile dotplots” (see Figure 2 of this paper). It is a dotplot of quantiles from the predictive distribution that has the useful property that counting dots corresponds to estimating intervals from the predictive distribution.
For example, if there are 20 dots in the display, and I count up 2 dots from the left and look at the time at that point on the x axis, that point will correspond to a predicted 2/20 chance of me missing my bus (or 18/20 chance of making it). The idea is to help people make judgements like, “oh, I’d be willing to miss this bus one or two times out of twenty” and then decide when to get to the bus stop based on that.
Interesting — I see several rejected designs in that paper. Why did you ultimately go with this design?
Indeed, and there were many more that didn’t make it into discussion in the paper! A lot of HCI lies in generating a lot of ideas and then chucking them.
The design we settled on elicited more precise probability estimates from people. It also fulfills several unaddressed needs that we found people had in a survey of existing users — things like answering the question, “what is the chance my bus has gone by already?” People unfamiliar with OneBusAway might think that’s an easy question to answer for a system designed to track the location of buses, but it turns out there is plenty enough error in the system to make that hard to answer.
In terms of rejected designs, we tried some crazy things, like visualizations of clocks with uncertainty and animations of buses arriving. The use of animation in conveying uncertainty is something Jessica Hullman (one of my collaborators on that paper) has worked on more; I really like that approach in general, but it has some problems in the context of real-time transit prediction: If you’re standing there trying to quickly decide what bus to take, you don’t really have the time to watch several animations.
So we ended up focusing back on static visualizations, and that led back around to the question of how to efficiently and effectively communicate discrete outcomes in a small space that can be read quickly, and that led around to developing quantile dotplots.
On your website, I saw some more work about communicating uncertainty in a bathroom scale application. I’d love to hear about that.
The weight scale work is similar in spirit to the bus arrival time prediction work. Scales tend to throw numbers at people — one data point at a time — and hope that they can make sense of their data. Even the best scales (that upload data to the cloud and provide graphs of trends over time) typically still just show one data point at the moment of weigh-in. On top of that, they provide no sense of the error in the data — how precise the scale is, or how much you can expect your weight to fluctuate.
I’ve analysed online reviews of scales to see how this affects people’s attitudes towards their scale, finding that statistical misconceptions can cause people to think the scale is worse than it is. But the “state of the art” of showing people one data point at a time does nothing to help guide people away from these misconceptions or help them get more value out of their data.
I’ve been looking at ways to improve this situation, partially through modeling a person’s weight (so we can explain some sources of bias and variance to them) and partially through redesigned scale interfaces.
This also sounds like a major win to me. Are you working on anything else?
Another area I am just starting to work in is building usable tools for Bayesian analysis of experiments. In that area I am focusing on researchers that you might call the “t-test” (or “SPSS”) researcher: someone who today probably just runs a t-test or ANOVA to analyse the results of their experiments.
Especially in HCI, these types of researchers could benefit from incorporating prior knowledge into their analyses using Bayesian approaches in order to get better estimates of the effects they are trying to measure (“better” in the sense of “less error”). While Bayesian analysis is becoming more widespread amongst a certain set of applied researchers, those tools aren’t usable for the typical “t-test” researcher (or alternatively, the researcher who throws their data into SPSS or JMP [statistical analysis software] and wanders around until a test result comes out).
The problem is that the existing tools designed for these researchers are not doing an effective job of guiding them to more appropriate techniques. That is, I don’t think the problem is with this user group, but that the tools available to them aren’t doing a good job of catering to their needs. I want to build tools that target that researcher, helping them set informative priors on simple models, and guiding them in the interpretation of their results.
Interesting – In this way your research could increase the effectiveness and quality of other research. How much of the difficulty of using Bayesian approaches do you think is related to tooling rather than it being difficult to build good models? How are these users’ needs different?
Bear in mind I am just starting out this particular research program, so I don’t have a lot of answers yet.
For the groups I am interested in— let’s say HCI researchers, though that is only a subset— a lot of the research questions they are interested in are something like “is there a difference in [some measure] between [technique A] versus [technique B] on [some task].” The traditional way to answer that question might be a null-hypothesis significance test: Is there statistical evidence to reject the hypothesis that there is no difference?
However, this null hypothesis is often obviously false: If I give people two different interfaces for some task, I would be surprised if there were exactly zero difference in performance between them. What researchers are often really interested in is how big the difference is, or how reliable.
This requires a shift from thinking about looking for differences to thinking about measuring differences. So a lot of what I will be interested in working on will be helping guide researchers through thinking about measurement instead of null-hypothesis significance testing. How that exactly works and what the interfaces might look like is a further-down-the-road set of issues.
Is there anything else that you’re particularly interested in?
I had a side project a while back looking at gender representation in image searches of occupations. That was a fun collaboration with two friends of mine, not core to any of our normal research.
In a research talk, one of my friends had observed a particularly stereotyped use of an image to represent an occupation being discussed in the talk. Presuming the image was pulled from an image search, we decided to get a bunch of Google Image search results and compare them to data from the Bureau of Labor Statistics to see how well gender proportions matched up.
We found that image search results don’t do a completely terrible job of representing real-world gender proportions in occupations, though there is a stereotype exaggeration effect, where (say) a search for CEO, a male-dominated occupation, will tend to have even more males in it than if it strictly reflected reality.
There’s also something we dubbed the “sexy construction worker” effect, where results from an occupation against the gender stereotype (say female construction workers or male nurses) will tend to be rated as less professional. That got us some press, including one article that noted the first image of a woman in the search results for “CEO” is CEO Barbie (unfortunately, that still seems to be the case).
Neat, do you have any of these articles?
Woah, that’s awesome! Moving on — you just finished your Ph.D. at UW. What are your plans now?
I’m very excited to be starting as an Assistant Professor at the University of Michigan School of Information in the Fall! I have actually moved to Ann Arbor already, which was something of a grueling process but is mostly complete now
Congratulations on the faculty position! Were you out during your job search?
Yes, I was. The decision to be out during the search was easy, since I have been out since before the beginning of grad school and would have no intention of going back in to get a job. Put another way, if I had to go into the closet to get a job somewhere, then I wouldn’t want to work there.
Do you think that this impacted your search at all?
It has impacted my search in the sense that there are cities I wouldn’t want to live in because I am gay. On the other hand, places that aren’t friendly to LBGTQ folks tend also not to be places I want to live for other reasons, like not having an art scene or not having much in the way of bars or restaurants. So I don’t know how much that changed the places I wanted to apply.
You’ve probably heard (and experienced) the saying that coming out is a continual process, not a moment in time. Sometimes when meeting someone new I will deliberately drop a mention of my boyfriend in the conversation, if some reason to mention him hasn’t already come up naturally.
But the funny thing about the academic job search is, if you are gay and in a relationship, there is a prescribed moment in the interview process to out yourself. During an interview you will have a meeting with the Dean or Chair of the department. Even though it is technically against the law, they will find some way to ask you whether you have a two-body problem. At that point I can mention my boyfriend, who is an artist, and that consequently I don’t have a two-body problem in the academic sense. And the nice thing about everywhere I interviewed was that that second piece of information — what he does, and how it might affect my decision to move somewhere — was the important piece of information to them.
But mostly the fact that I’m gay came up naturally in conversation anyway. A lot of the job search process is a two-way interview where everyone is trying to decide if they might get along as colleagues for years. So you have talk about work and play, and significant others come up naturally.
Are you planning to be out to students?
Definitely. I’m not sure it is possible to even have that choice. After accepting the position at Michigan I was at a party at CHI. A Michigan Ph.D. student who I did not know came up to me and invited me to check out an LGBTQ reading group when I got into town. I was a little surprised he already knew I was gay. Not that I minded! Quite the contrary, it was a nice gesture that made me feel very welcome. But the point is, I don’t think I could be out to professors in the department and not out to students, even if I wanted to.
What are your thoughts on people talking about it? It’s tricky because you never want to accidentally “out” someone, but at the same time, it’s nice for people to know who you are if you are comfortable with it.
Right. Part of really being out of the closet is letting go of control over who knows you are gay. Otherwise — as someone once put it to me — you aren’t out of the closet, you are just pulling other people in with you.
So if I bring up my partner or my sexuality casually in conversation with someone else, I should expect them to mention that fact to another person in any situation they might talk about a straight person’s significant other. It’s equality in gossip: I will hear the odd tidbit about a straight colleague’s husband or wife or children, so I should expect others to hear the equivalent tidbits about me.
I recognize that some people are not comfortable having others out them, and in some cases I might be guarded due to legitimate concerns about safety. But if you are gay and that fact comes up casually, I think it should be fair game for others to talk about it unless you explicitly say not to. You aren’t comfortable with your identity until you are comfortable with other people talking about your identity, and the environment you are in isn’t a totally equal one unless gossip is equal opportunity. The alternative is “don’t ask, don’t tell.”
What was it like for you being out in grad school?
Being out in grad school was great! I was out from the beginning and never had any issues because of it. I have found both Seattle and UW to be a great place to be gay. Within the department, I knew several other grad students who were LGBTQ.
There were some LGBTQ groups on campus as well, though I never got very involved in them. I signed up for the RainbowGrads mailing list, though I never went to any of their meetings — by the time I discovered them, I already had friends and a support network, so I didn’t see the need for myself. I went to an oSTEM meeting, but it was aimed mostly at undergraduates, so I didn’t get much out of it.
I think in general the support systems are there if you want to take advantage of them, but I was already at a good place in my life with respect to being gay so I didn’t really have the need.
I’m glad your experience was great. You say you have a great support network; how did you find such a network?
I found it mostly through other grad students I made friends with. The department is fairly social and has plenty of ways for people to meet each other, especially during your first few years, and I built up my friend group that way.
Did you have any role models who were LGBTQ?
Not in particular. The LGBTQ professors I know are mostly people who I was friends with when they were Ph.D. students, so they weren’t really role models so much as friends and colleagues.
Do you think it would have helped you to have LGBTQ role models? Or do you think it’s enough to know colleagues?
For me it was enough to know colleagues and friends (I would stress the friends part). But this sort of thing varies a lot by individual. It was more important to me to have friends I could talk to about LGBTQ issues.
I think for some people it is more important to them to have role models to aspire to, to be able to see that a person like them in certain ways can do something they want to do. That can be very empowering. It could be my general disposition, or it could be part of my privilege as a white male that I didn’t really feel the need for role models. Different people find support and motivation in different ways.
Different sub-communities of computer science have different cultures. What has your experience been like as an LGBTQ HCI researcher?
It has been completely welcoming! I know several other LGBTQ grad students in HCI (actually, most of them are now faculty or researchers in industy).
At some conferences, people have organized LGBTQ parties (sadly they have always conflicted with other things for me, so I haven’t been to one yet). This year for the first time CHI hosted a huge Diversity Lunch at the conference. I attended that and had a great time. It included small group discussions and a variety of invited speakers, including at least one gay speaker. There is even research into LGBTQ issues conducted within HCI.
That is great to hear! Are orientation and gender identity things that can be freely discussed within HCI, or are they considered private?
Definitely freely discussed! Even outside the lunch that was basically designed as a safe space to discuss such issues.
What sort of HCI research delves into LGBTQ issues?
I’m not intimately familiar with it since it isn’t my sub-area, but I have probably seen a few papers on it each year in the CHI program. There is a lot of work at CHI studying how different groups of people use social media, and I think most of the LGBTQ research I’ve seen has been in that realm. For example, there was a paper this year called “LGBT Parents and Social Media: Advocacy, Privacy, and Disclosure during Shifting Social Movements” that got an honorable mention (oddly enough, a Michigan paper).
Do you have any ideas for why HCI has such an open culture?
The expected response for any question about why HCI is different from other subfields of CS is to point to the fact that HCI is the intersection of CS with fields like psychology, design, and sociology (to name just a few). I think the combination of academic cultures going on there tends to pull HCI culture in a slightly different direction from other areas of CS. I suspect that explanation applies here.
Thanks for your time!
Thank you to Pavel Panchekha for consulting on the interview about Bayesian analysis.