Twitter Should Use Random Sample Voting For Abuse Reports 132
In August, Twitter user Kristin Puhl made public the fact that another Twitter user had tweeted at her:
f@#king die feminist moron i'm coming after u and raping u.
and when Puhl filed an abuse complaint with Twitter, Twitter responded after two days:
We've investigated the account and reported Tweets for violent threats and abusive behavior, and have found that it's currently not violating the Twitter Rules (https://twitter.com/rules).
(The "rules" linked in the message include the clause "You may not publish or post direct, specific threats of violence against others.") Twitter must have changed their mind eventually, because the account of the user who sent the message is now gone, but why didn't they close it the first time?
Twitter can't effectively adjudicate all the abuse complaints that they get, but I don't blame them. I don't think they publicize numbers for how many abuse complaints they receive every day, but I'm sure that it's more than an internal review panel could handle fairly. Twitter should not be faulted for that. They've created a world-changing tool, and they shouldn't have had to stifle the growth of their platform just because it grew faster than their ability to handle the abuse reports.
But now that they're publicizing their latest tools for handling online harassment, it's fair to ask more of them. And while the tools may streamline the process of categorizing incoming abuse reports, there's always going to be a human review bottleneck, which will get tighter as the Twitter platform continues to grow.
So I'd suggest the same solution that I suggested for Facebook abuse reports: recruit a pool of volunteers from the general public to review "abuse reports". (You would need a "critical mass" of at least tens of thousands of reviewers for my idea to work, but Twitter shouldn't have trouble amassing that many people for a special program.) Then when an abuse report comes in, do the following:
- Some small number of reviewers -- say, ten -- are randomly recruited from the pool of volunteers.
- Each of them looks at the reported content and the category of abuse that it was reported under, and votes Yes or No as to whether the content meets the criteria for abuse.
- If some threshold of users (say, eight) vote that it does, then the report gets bumped up to a higher-level review. This "higher-level review" could mean having a new, larger pool of users (say, twenty) look at the content and vote on it, in case the original eight-out-of-ten vote was a statistical fluke. Or it could mean forwarding the reported tweet to some human review panel at Twitter -- which now has far fewer abuse cases to review, because it only has to look at the reported tweets that cleared the hurdle of getting eight out of ten votes for violating the guidelines.
These numbers are just guesses. I might be over-optimistic about how many reviewers would even respond when Twitter asked them to vote on whether some content was abusive (even though that's what the reviewers signed up to do) -- it might turn out that to get even ten responses, Twitter would have to nag 50 people to come and vote on a piece of content. And the size of the voting initial voting panel should be large enough to avoid statistical flukes most of the time -- if a tweet is inoffensive enough that only 10% of the reviewer population would consider it "abusive", you'd have to be really unlucky to convene a panel of 10 users where 7 out of 10 voted to label the tweet as "abuse".
As long as the size of the reviewer population grows in proportion with the Twitter user base (or, more precisely, as long as it grows in proportion with the volume of abuse reports coming in), this system scales as much as you want it to. (Well, unless the "higher-level review" involves review by an internal panel at Twitter, which still creates a bottleneck.)
Because the voting panel is randomly selected from among the entire pool of volunteers, that means you can't "game the system" by forming a mob with dozens of your friends so that everyone can file an abuse report about the same content at once. As long as your mob only comprises a tiny proportion of the 100,000+ reviewers in the system, there's virtually no change that a randomly selected panel would contain enough of you to swing the vote.
This could also potentially result in an almost-instant turnaround time for handling abuse cases (a matter of reassurance for victims of normal harassment, and a matter of life and death in the case of suicide threats or threats of violence). Twitter could restrict their random sample to only those users who happen to be signed in at the present moment, and who have a minute or two to review a piece of content and vote on whether it violates the guidelines.
Tweets are by definition public, so there wouldn't be any potential privacy violation in taking someone's tweet, putting it before a panel of 10 volunteer reviewers, and asking them to determine if it violated the terms of service. Direct Messages sent via Twitter, on the other hand, are intended only for the recipient, and are not public by default. If a recipient wanted to flag a Direct Message as abusive, they would have to specify whether they want the content to be reviewable by a panel of randomly selected public volunteers. So in the case of the tweet received by Kristin Puhl -- "fucking die feminist moron i'm coming after u and raping u" -- even if she had received it as a Direct Message from someone she was following (you can only receive DMs from someone if you're following them), presumably she would have been OK with showing the tweet to a panel of volunteers, who probably would have voted that it was in fact abusive. On the other hand, sometimes a user might receive abusive DMs where they want to report the abuse, but the DMs might contain sensitive information that they don't want publicized to randomly selected volunteers. So those abuse reports might have to be handled the old-fashioned way at Twitter, by internal review, which still creates a bottleneck. But hopefully the abuse reports about Direct Messages comprise only a small minority of abuse reports that Twitter receives, since most talk about abuse on Twitter comes in the form of public tweets. (If someone is "abusing" you via DMs, you can just unfollow them.)
Twitter could even be completely transparent about the entire voting process: "Your complaint has been reviewed by 10 people. 8 of them agreed that the tweet in question violated our guidelines. This is above our minimum threshold of 7 that triggers a higher-level review of this content." (Twitter presumably wouldn't want to tell the complainer who the voters on the panel were, since the complainer might harass the individual voters if the voting panel as a whole rejected the complaint. But there's no reason not to be transparent about the actual numbers.)
Why would someone sign up to volunteer to review abusive content? Maybe for the glimpse into strangers' lives. Maybe hoping to save copies of some of the porn contained in the tweets that get reported for abuse. (Of course, there are easier ways to get porn online, but maybe they get off on the fact that some particular pornographic image made someone angry and upset enough to report it.) Maybe they altruistically believe it's part of their civic duty towards the Twitter community. Maybe because they're bored.
Whatever people's myriad motivations for signing up, the important thing is that there's still a statistically significant difference between the number of "yes" votes received when content truly is abusive, and when it's not. Even if you have people signing up as reviewers for all kinds of weird reasons, a tweet like "fucking die feminist moron i'm coming after u and raping u" is still going to receive, on average, more "yes" votes than a tweet like "I respectfully disagree, so let's go our separate ways".
If Twitter were nervous about rolling out a system like this, ceding control of the abuse-report-handling process to a pool of volunteers, they could always do their own random sampling of the random-sample-voting system, to see how it was working. An internal auditor could pull 100 of the abuse report cases that have been handled by the random-sample-voting system recently, decide in each case whether the tweet did in fact violate the abuse guidelines, and then look to see if the voting system reached the same answer. As a control in the experiment, look at some abuse reports that were routed to the old-fashioned internal review panel during the same period, see how they handled the reports, and see how they fared in comparison. I would confidently bet money that the random sample voting system would handle the abuse reports more accurately, and faster, as well.
This won't do much to deter abusers who create an endless series of throwaway accounts for harassment purposes, which makes it futile to block or report any particular account. But it would at least get step zero right, which is to correctly adjudicate whether a tweet is abusive or not. And it would do it in a way that is scalable, non-gameable, and transparent. Plus a few volunteers would get an interesting story to tell at dinner.
Fuck what Bennett Thinks. (Score:5, Insightful)
Re: (Score:2)
We so desperately need a censorproof, indelible internet.
Freenet is there, waiting for anyone to use it. It may be no more anonymous than TOR (and slower), but being peer-to-peer there are no servers to seize.
I've always wondered whether it would be reasonably fast if it became popular for filesharing, but given most people's lack of patience it's unlikely we'll ever know. At least it's a place where Wikileaks could be immune to takedown, for whatever that's worth.
Re: (Score:2)
No doubt. That's the thing about an uncensorable network - if it can host stuff governments don't like, it's certainly going to host stuff I find appalling. That's the trade-off.
Re: (Score:2)
0.7 did not introduce any such weakness - it was always there. Darknet was the dev team (one student, mostly, though a bright guy), giving up on that issue, at least short term, to create something that would at least work a little in very oppressive states.
But it's not less anonymous than TOR - uploads have very strong anonymity, and while on paper it's easier to de-anonymize download on Freenet than TOR, in practice we know of NSA tools that work for TOR and not for Freenet (which may just be what got le
Re: (Score:2)
The internet is just another artifact of society as a whole, and increasingly just a part of the military-industrial complex's mass pacification programme.
Re: (Score:1)
Re: (Score:2)
When the first 50 posts, all at +4/+5, all say essentially the same thing I did - Slashdot has no "plausible deniability" here. No editor can approve Benny's crap with a straight face at this point - Anyone putting his crap on the front page at this point does so solely to troll the community.
Who the fuck is Bennett Haselton (Score:3, Insightful)
Re:Who the fuck is Bennett Haselton (Score:5, Funny)
I hear he's a frequent contributor.
He apparently doesn't even meet the requirements for becoming a Slashdot editor. Since the average /. editor is slightly less intelligent than bread mold, that tells you a lot about Bennett. Why would anybody pay attention to him?
Re: (Score:2)
I hear he's a frequent contributor.
He apparently doesn't even meet the requirements for becoming a Slashdot editor. Since the average /. editor is slightly less intelligent than bread mold, that tells you a lot about Bennett. Why would anybody pay attention to him?
On account of his awesome magnetic poetry [magneticpoetry.com] skillz, naturally!
Re: (Score:1)
He is a frequent contributor.
Re: (Score:1)
Re: (Score:2)
He has never denied being involved in the notorious "Black Dahlia" case in 1940s LA.
But above all, he's a Regular Contributor.
Re: (Score:1)
could be easy (Score:2)
if a post gets multiple reports it gets disabled, and the user gets a notification that it was disabled, why, and a change to reply stating why you believe its wrong, for a manual review.
Re:could be easy (Score:5, Insightful)
Are you crazy? Lots of places--most notably comment boards on news sites of every political stripe--are known for having a bunch of sockpuppets who collectively flag posts of people they don't like. Sheer volume lets them silence opposing opinions.
That's one thing Slashdot got right with metamoderation...probably people who are known to do that, don't get mod points again, and only active users get mod points at all (so sockpuppeting takes more effort than you get benefit).
Re: (Score:2)
Where slashdot got it wrong. (Score:2)
Yep. Which is why /. should require that every down-mod be accompanied by a short explanation of WHY it fit "abusive/trolling/offtopic".
Up-mods don't matter. If you want to mod something up then no explanation is necessary since they don't "bury" unpopular opinions.
Re: (Score:2)
Yep. Which is why /. should require that every down-mod be accompanied by a short explanation of WHY it fit "abusive/trolling/offtopic".
I believe all Troll mods of roman_mir posts are automatically justified. He can't be doing anything but trolling, at this point.
Re: (Score:2)
If everyone gets the option to rate one post per day, they can afford to spend some time thinking about it, avoiding that trap.
If everyone gets the option to rate every post, ratings will be dominated by those who use the right every time - vote on everything. Surprise surprise, people who spend one second per post won't be especially thoughtful. Thus, they will abuse it as an "I don't agree" button vastly more.
Responsible people's judgements take too long time. They just can't compete with impulse voters.
Re: (Score:1)
Hey! You forgot the trigger warning warning. Your trigger warnings are setting off my trigger you insensitive clod!
Re: (Score:2)
It's the rabid extreme right wing bigots, racists, elitists and misogynists who moan most about people trying to censor them when they read a comment critical of their ramblings.
Re: (Score:2)
It's the rabid extreme right wing bigots, racists, elitists and misogynists who moan most about people trying to censor them when they read a comment critical of their ramblings.
The [string of ad hominems]s here are a larger group of people who are tired of being labeled [string of ad hominems] because they don't think you alone should be able to declare who deserves "mutual respect" and "tolerance" and who doesn't. Nor that you should be the arbiter of whose ramblings deserve criticisms (not your own of course!) Apparently you think a lot of things deserve to be insulted (not too respectful really) including being "right wing".
Never have I seen such a succinct example of why peop
Re: (Score:2)
if a post only gets 1 report - manual review
if a post gets multiple reports it gets disabled, and the user gets a notification that it was disabled, why, and a change to reply stating why you believe its wrong, for a manual review.
That is a bad idea and one easily abused. 4chan alone would be shutting accounts down right and left.
Re: (Score:2)
Re: (Score:2)
just disabling said post would do the same effect.
if you did that, then twitter wouldn't have any posts left at all.
and you know what kind of people would volunteer to do the voting in the proposal? fucking dickwads. it's not like a normal twitter user wants to spend his/her time moderating the service.
Re: (Score:2)
Ah yeah, a scheme you just cooked up yourself, without justifying it. Never mind that it has terrific abuse potential, and even in the long term relies on a fixed, privileged few (the reviewers) being fair.
Popular Shalshdot Opinion (Score:4, Funny)
Re: (Score:1)
As long has he generates hits and comments he will be featured. You'd be better just ignoring his articles.
As for the content control, I suggest they use a system like LoL. It works for them I'm sure it would work for Twitter. The only thing I'm not sure of is how they would reward the contributors.
Re:Popular Shalshdot Opinion (Score:4, Insightful)
As long has he generates hits and comments he will be featured. You'd be better just ignoring his articles.
Not if people are just coming to complain, trolling your users isn't a sustainable model (well it is, but only if you make a 3rd party the direction of their rage).
That being said Bennett's proposal is fine, it's just not what people come to /. for. The same thing happened when they employed Jon Katz [wikipedia.org] as a writer and his essays were regularly met with ire. /. calls itself a blog but it's really a news aggregator with some limited commentary. There's a big difference, people read blogs because they enjoy the author's voice and reputation, this is particularly true in the case of longer essay style posts, it's not even the quality of the writing as much as a matter of persona taste. There's no reason to think that Bennett fans and /. readers are groups with significant overlap which is why he lacks fans.
But more importantly there's a status issue, everyone who writes a comment is fighting to get their voice heard and is playing by the same rulebook. No matter how inciteful or amazing your comment the best you're going to get is a +5 hidden in the comments section*. People are going resent the fact that Bennett seemingly feels his musings are so insightful they deserve their own story, the editors deciding to agree with him doesn't help.
* There was a short lived /. experiment where they'd choose a handful of comments from a story then post a new story based around those highlighted comments, presumably as a way to honour top comments^H^H^Hrestart the discussion that had gotten so many page views. One of my comments chosen for one of these stories (I think it might have been the first one they did), which would be cool if the experiment lasted more than a couple weeks.
Re: (Score:1)
Re: (Score:3)
I guess what I was getting at is that the users who comment in a non constructive manner such as the parent comment should not be modded up. Instead, we the readers should be able to mod down the writer of the article. If a writer is bad enough they will disappear off /.
Re: (Score:2)
I doubt I'm in a small minority, but I don't read Slashdot as a news aggregator (which it's lousy at), but for the comments. The signal-to-noise ratio isn't what I'd like, but some of that signal is really, really good.
Re:Popular Shalshdot Opinion (Score:5, Insightful)
Re:Popular Shalshdot Opinion (Score:4, Insightful)
I don't care about Bennett's opinion on a great deal of things, and I really want to not click on his posts to NOT generate traffic - hopefully thinking that he will eventually if no one is reading his posts. But then, I REALLY like to see people make fun of him in the comments, so I click on the post. But then that adds page views to the post meaning Bennett will do more of these in the future. It's a vicious, ugly cycle. I want to break it, but it's so much fun to make fun of Bennett....
So you're part of the problem.... and unfortunately, so am I.
Re: (Score:1)
We are all Bennett Haselton. Oh god, please kill us.
Re: (Score:1)
I don't care about Bennett's opinion on a great deal of things, and I really want to not click on his posts to NOT generate traffic - hopefully thinking that he will eventually if no one is reading his posts. But then, I REALLY like to see people make fun of him in the comments, so I click on the post. But then that adds page views to the post meaning Bennett will do more of these in the future. It's a vicious, ugly cycle. I want to break it, but it's so much fun to make fun of Bennett....
So you're part of the problem.... and unfortunately, so am I.
Me too, but we have to get the word out about the petition: http://petitions.moveon.org/sign/drop-bennett-haselton [moveon.org].
Re: (Score:1)
Slashdot have given this guy a megaphone. He will not go away on his own, so the best thing to do is call him out in the comments. Otherwise this place will turn into a hugbox and they'll start a anti-Geek-Krystalnacht here too.
On second thought when he's gone, they'll replace him with a female writer on the same issues and any criticism of the articles will be labeled "misogynist" and Slashdot will start deleting comments anyway. Such is the march of the modern internet.
Re: (Score:2)
I hear ya, man. I figure I could give up Bennett at any point. Snap, just like that. But do I have to give up Bennett? Is it really so bad? Sure, I'm wasting time, and sometimes after reading Bennett I temporarily forget how to work the thermostat or what the Judicial branch does, but I only do it now and then, there's no problem.
Yeah, I can give him up any time. When I'm ready to, I'll just read one more Bennett regurgitat
Re: (Score:2)
It just encourages people like everyone's favourite Regular Contributor.
Re: (Score:1)
More dribble (Score:1, Insightful)
We don't care what he thinks!!!
We think he should get his own blog!!!
Re: (Score:2)
The period always goes inside the quotation marks. Never outside.
Slashdot needs an abuse mechanism... (Score:5, Insightful)
Re: (Score:1)
Just sign the petition: http://petitions.moveon.org/sign/drop-bennett-haselton [moveon.org].
Re: (Score:2)
I'm curious what frequent contributor Bennety Assleton thinks about this petition. Usually, he can summarize a complex issue succinctly, and convey useful information quickly. I'd love his take on this.
Also, some other topics:
Fart ordering- quick sort or bubble sort?
What equals 6, emacs or vi?
What's life like being consummately oblivious?
How many sticks are up my ass? (A Slashdot pole)
Deadly auto-erotic asphyxiation methods, a hands on review.
Re: (Score:2)
Dude, we got a twofer! A Bennet post AND a Nerval's Lobster post in one day!
How much is Bennett paying for this mouthpiece? (Score:2, Insightful)
I'm starting to think that he's a big part of the funding keeping Slashdot afloat, by paying them to publish his stuff.
No (Score:4, Insightful)
No
Yup (Score:2)
Long have I waited for this moment (Score:1)
Now I may bathe myself in the wisdom of this post.
Got as far as "Bennett Haselton writes:" (Score:3, Insightful)
Re: (Score:3)
Re: (Score:2)
Doesn't always work (Score:2)
For example, a majority of US whites think certain cops are innocent, but a vast majority of US black think the cops are murderers.
Which side is right?
Even when there's video of the cops actively killing unarmed people, nobody gets charged.
But if we use your system, someone saying the cops are murderers will be reported and sustained as "abusive", given that Twitter is mostly used by white people.
Same thing goes with global warming. 87 percent of Americans think it's happening now. But the 17 percent that d
Wow! (Score:2)
Think of all the problems that could be solved with unpaid labor!
Re: (Score:2)
Why should suicide threats be alerted?
Yeah, how dare anyone interfere with the right of mentally ill people to fuck themselves up?
It's literally Hitler.
Yea Bennett Haselton! (Score:1)
Actually, I *always* read Bennett's posts. The comment stream is, by far, the funniest postings on slashdot. I'm delighted by the myriad inventive ways slashdotters have discovered to mock this maroon.
If Bennett did not exist, it would be necessary to invent him.
Who? (Score:2)
What kind of people are those going to be who volunteer to do a corporation's job?
Those with an agenda. (Score:2)
That would be those people who already have an agenda that they believe could be furthered by restricting other people's accounts.
Tyranny of the majority.
And that isn't counting hiring people to do that. For just $X a day, you can down-vote post opposing Y and up-vote posts supporting Y. Think about whatever political position you don't like and imagine those people doing that.
Bennett Haselton is an idiot. That's okay.
The fact
This shit belongs on Medium (Score:3, Insightful)
Re: (Score:2)
What is it about Medium? It's like a forum for bland idiocy, which admittedly is a step up from the kind of idiocy we see elsewhere on the 'Net, but it's as certain as Bennett to produce garbage.
Re: (Score:2)
Re: (Score:1)
So sign the petition and let's get him dropped: http://petitions.moveon.org/sign/drop-bennett-haselton [moveon.org].
Re: (Score:2)
Fuck off (Score:2)
Filter error: You can type more than that for your comment.
Re: (Score:2)
Or because they want to feel helpful. Or because they like watching numbers grow higher. Those are both perfectly legitimate reasons I was happy to hit 2000 rep on SO, so I could start helping with the Low Quality Posts review queue there (and the reason I currently have well over a thousand helpful flags).
"it's fair to ask more of them". Really? (Score:3)
What are you going to do, Bennett? Call and ask for your money back? Demand to see life's manager and return the lemons?
What's original about your idea that you don't think the folks at Twitter don't already know about from places like Slashdot, Reddit, Google, etc? At this point, crowdsourcing volunteers by one method or another to help rate the quality of something isn't exactly an unknown way of doing things. Do you have any original ideas?
So what's the problem? (Score:1)
f@#king die feminist moron i'm coming after u and raping u.
From personal eye-witness accounts of vile feminists and SJW's, I think this twitter comment is actually fairly tempered and reasonable. The poster should be given a medal for his restraint.
Why can't I kill this garbage? (Score:2)
Dear /.: please enable killing all Bennett posts without completely disabling the site. If I put "Bennet Hasselton" in my list of filtered terms every story disappears.
Riot beat Bennet to the idea (Score:2)
Riot has had a system like this for some time, the Tribunal [leagueoflegends.com], and they allege it works pretty well. It used to have an in-game based reward (absolutely minuscule amount of IP, the in game currency), but they have since removed it, and last I checked it still had high numbers. I don't know if Riot is the originator, but I know it's a pretty major part of their abuse/harassment control.
I really don't know if a Tribunal style method would carry over to Twitter - I remember that part of the reason that people li
Bennett is our Common Distaste (Score:3, Interesting)
On a TF2 server I regularly play on (I only play on other servers if it's empty or on a map I hate), we have about 40-50 regulars that are there at least once a week, if not multiple nights, for many hours. One of these was a guy who was usually getting on at least one person's nerves every night, but he's been absent for a few weeks, likely just busy. In the meantime we have a new person who has taken on the same role. He's a dick, but has some intelligence and is never a *huge* dick (at worst one or two people will try to votekick, but most who find him annoying have just muted him.)
Last week one of the old regulars, also an admin on the server, was on at the same time and was telling the new person that the admin appreciated his presence, because there was universal annoyance, at best, amongst the server population that helped bind them together. It was something like a common cause, but replacing productivity with hate. While it was certainly intended to be a riff on the dick, there was some truth to what the admin was saying.
Perhaps that's what Bennett is to us. The whole Beta thing has really died down (or I've willfully ignored it), and that was a very uniting aspect of millions of /. users. Since it's died down, these "stories" by Bennett Hazelton have begun. Perhaps these aren't intended to be actual stories, but to give a "common distaste" (or detest, if you prefer) amongst /. users that will act as a common ground: /. can act towards a large goal, fueled by our mutual hate for Bennett blog posts. A grand conspiracy by /. editors (and Dice?) for the greater good.
A: "Hey fuck you ignorant conservative"
B: "No fuck you, lazy liberal"
A: "Ah man, it's another Bennett article. I hate that guy's drivel."
B: "Oh, really? Me, too. Wish I could ignore all of his articles."
A: "Heh, yeah. So, hey, about earlier..."
And thus
Re: (Score:1)
Take a stand and sign the petition: http://petitions.moveon.org/sign/drop-bennett-haselton [moveon.org].