Programmatically Capturing Spam with Detextion.io

There is a [modified] adage about attracting users to web apps.

"Build it and spammers will come."

The fact is that if your users can enter text others will see, your app is going to be a target for spammers. This means comments, profiles, messages, feeds and even usernames. Build it and spammers will come.

Despite this being common, preventing it is a pain in the ass. Human moderators are the usual answer. But sending messages through a human gatekeeper has some problems. Paying the moderators is expensive and immediate feedback on the site is delayed while a message is reviewed. Worse still is if you are the one stuck reviewing all that content. Trust me, you ain't got time for that.

Another avenue is to ignore spam until your users complain. This may be easier on you or your team, but it kills your app integrity. As soon as users identify spammers, they start to wonder how much of the activity on your site is fake. Their estimates will not be charitable.

These concerns amount to the death knell for some products. User trust is sacrosanct. Once diminished, rebuilding a user's trust is one of the hardest things for a website to do.

Achieving automated moderation

Paying moderators is an expensive pill to swallow for many early-stage companies. It is basically a non-starter for blogs, forums and side projects. Automated moderation, should it be achieved, would make eliminating spam a reality for smaller teams. Move fast and don't sacrifice content quality, as it were.

This was the drive behind Detextion. It is built with years of experience in text analytics, online dating sites and social applications.

Detextion is an API capable of identifying trends associated with spam. This includes emails, phone numbers, web urls, etc. it returns its findings in a categorized breakdown. Your app gets to choose what is, or is not, allowed in user submitted text.

Implementing Detextion is a matter of making a secure POST. In ruby:

def moderate( text )
	uri = URI.parse( 'https://www.detextion.io/analyze )
	detextion = Net::HTTP.new( uri.host, uri.port )
	detextion.use_ssl = true

	post = Net::HTTP::Post.new( uri.path )
	post.set_form_data( {
		api_key: 'your_api_key',
		message: text
	} )

	detextion.request( post )
end

Of course, which filters your app respects is specific to each use case. To give you an idea of how this works, here are a couple of basic examples at programmatic moderation.

Programmatically moderating comments

One of the more common abuses of text fields is the spamming of contact or website information. Detextion recognizes emails, urls and phone numbers. Perhaps the easiest filter we can write is to automatically delete comments with phone numbers in them.

# Run this through Detextion...
"Would you like to see me tonight? 2 oh 3 four 5 6 seven 778"

# and it will return:
{
  "filters": {
    "phone_numbers": [
      "2,oh,3,four,5,6,seven,7,7,8"
    ]
  }
}

Detextion categorizes and labels its findings. This means we can react to phone numbers differently than text with emails or website urls.

comment.text = "Would you like to see me tonight? 2 oh 3 four 5 6 seven 778"

results = moderate( comment.text )
if results[ 'filters' ][ 'phone_numbers' ].present?
	# Hide instead of delete (more on this below)
	comment.hide()
end

Censoring profanity

Another useful application of Detextion is to censor posts. Consider the following.

# Input
"i'm tired of these fucking swears in my motherfucking posts."

# Result
{
  "filters": {
    "profanity": [
      "fucking",
      "motherfucking"
    ]
  }
}

We can work with this. By applying a regex, we can replace any identified profanity in the submitted text.

comment = "i'm tired of these fucking swears in my motherfucking posts."
results = moderate( comment )

results[ 'filters' ][ 'profanity' ].each do |swear|
  comment.gsub!( /\b#{swear}\b/, '****' )
end

# => "i'm tired of these **** swears in my **** posts."

Control your content quality

These sections make up just the start of what you can do with Detextion. What your app considers an acceptable bar for quality is different from many others. Detextion finds everything from emails to social networks to racial slurs. It gives your the information. You get to decide what passes as high quality.

Detextion is my step towards improving all of our app's content quality. As someone who strives for efficiency, the idea of a human gatekeeper in my projects is an absolute no-go. Detextion solved this problem.

As a final note i would like to share my biggest learning after years of battling spammers: do not show them that their content is being moderated. It will save you a world of headache and it will protect your users' experience immensely.

Our apps survive because our users trust them to provide value. As soon as customers start thinking activity is generated by spammers, our app is as good as Ashley Madison. Fucked.

{ "filters": {"profanity": ["Fucked"]} }

tl;dr: Detextion automatically identifies text from spammers and prevents them from ruining your app's integrity.