Twitter Face Filter Bot REST API System

Twitter API and Tweet Retrieval

The REST API system was built using Spring Boot in Java. The basic tweet retrieval code makes use of the Twitter API through the use of the Java library Twitter4J. Once configuring and authenticating with the Twitter API is done, a search is completed on Twitter using the Query class in Twitter4J as follows.

try {
  // Create query object, perform query
  Query q = new Query("@FaceFilterBot");
  QueryResult r = null;
  r = t.search(q);

  // Collect tweets into an ArrayList called statuses
  for (Status status : r.getTweets()) {
    statuses.add(status);
  }
} catch (TwitterException e) {
  e.printStackTrace();
}

Once all the tweets found are added into an ArrayList any media components within the tweets are able to be extracted using this getMediaURL() function I wrote.

public ArrayList<String> getMediaURL(Status s) {
  ArrayList<String> media = new ArrayList<>();
  String mediaURL;

  if (s.getMediaEntities().length != 0) {
    for (int i = 0; i < s.getMediaEntities().length; i++){
      if (!Arrays.asList(s.getMediaEntities()[i].getVideoVariants()).isEmpty()) {
        mediaURL = s.getMediaEntities()[i].getVideoVariants()[0].getUrl();
        //System.out.println(mediaURL);
      } else {
        mediaURL = s.getMediaEntities()[i].getMediaURL();
        //System.out.println(mediaURL);
      }

      media.add(mediaURL);
    }

    return media;
  } else {
    ArrayList<String> returnArr = new ArrayList<String>();
    returnArr.add("Text Only");

    return returnArr;
  }
}

This function loops through the media entities in a status using the getMediaEntities() function in the tweet passed through as Status s. It then checks to see if the current media entity being looped over is a video using by checking if getVideoVariants() returns an empty array to determine if the media currently being checked over has a video variants array. If it does not then it is not a video so can be retrieved using s.getMediaEntities()[i].getMediaURL(). Otherwise it retrieves the URL using the getURL() function on the first video variant.

Once these URLs are retrieved all of the Status objects are turned into objects of the Tweet class which I wrote. As can be viewed in the TwitterFaceFilterBot Github repository. This object is what is used to structure the final JSON Structured data that the REST API returns.

The Tweet class consists of a simple constructor that builds a basic object which holds the data from each tweet that the bot may need. This data comes down to the following:

• Text content.
• Text content with the link to the tweet removed. (When the Twitter API returns a tweet with media it returns the tweet with a shrunken link to the actual tweet attached to the end of the text content). This link is removed using the following piece of code which simply returns the substring of the text up until the start of “https://”.

   public String removeLink(String tweet) {
     try {
       return tweet.substring(0, tweet.indexOf("https://")).trim();
     } catch (IndexOutOfBoundsException e) {
       return tweet;
     }
   }
  

• The display name of the person who tweeted @FaceFilterBot
• The username of the person who tweeted @FaceFilterBot
• The ArrayList of mediaURLs found in the tweet by the getMediaURL function discussed earlier.

The following set of functions are the functions used in converting the Status Objects into Tweet Objects. It is pretty self explanatory however, a basic overview is the statusToTweet function creates and returns a tweet object for every status passed in through the status parameter. the getMediaType(mediaURLs.get(0)) function is a simple function I wrote which simply finds out the type of media the tweet includes by checking the first URL in the mediaURLs ArrayList and checks for a bunch of different file extensions within the URL to determine whether the bot is dealing with photos, videos, or no media. Plain text (no media) is returned as 0, where as photos and videos are 1 and 2 respectively. It checks if there is any media: if (_mediaType == 0) and if so it creates a Tweet object with null a null mediaLinks Object. Otherwise it creates a Tweet object containing the necessary mediaURLs.

public Tweet statusToTweet(Status status) {
   ArrayList<String> mediaURLs = getMediaURL(status);
   int _mediaType = getMediaType(mediaURLs.get(0));

   Tweet tweet;
   if (_mediaType == 0) {
     tweet = new Tweet(status.getText(), status.getText(), status.getUser().getScreenName(), status.getUser().getName());
   } else {
     tweet = new Tweet(status.getText(), removeLink(status.getText()), status.getUser().getScreenName(), status.getUser().getName(), getMediaURL(status));
   }

   return tweet;
 }

statusesToTweets() takes every single status retrieved from the twitter API and converts them into the Tweet Object using the statusToTweet() and returns an ArrayList<Tweet> containing all the new Tweet objects. mostRecentStatusToTweet() converts just the most recent status received or the one in the first index of the retrieved statuses.

 public ArrayList<Tweet> statusesToTweets() {
   ArrayList<Tweet> tweets = new ArrayList<>();
   for (Status status: statuses) {
     Tweet tw = statusToTweet(status);
     tweets.add(tw);
   }
   return tweets;
 }

 public Tweet mostRecentStatusToTweet() {
  return statusToTweet(statuses.get(0));
 }

SpringBoot and REST API System Deployment

Once the basic tweet retrieval system was designed I built and deployed a simple REST API service to retrieve tweets and return them when a request is made to a specific URL.

The SpringBoot REST API system works by having a base driver (Application Class) in order to launch the server system. For this project the Base class acts as this class and of course this is where the main method is located and is where the REST API system begins. Once the REST API server is online requests are made through the use of REST controllers. Controllers are essentially methods that are carried out when a certain request is received by the REST API service.

The controllers used in this project are just simple GET requests since the entire purpose of the REST API service here is to return the tweets received in a JSON format.

The two main controllers in this project are as follows (b is an instance of the Bot (the class containing all Twitter API interaction stuff and tweet retrieval stuff) class):

• /getTweets mapping. Returns all tweets collected in JSON format.

  @GetMapping("/getTweets")
  public ArrayList<Tweet> getTweets() {
    b.handleSearch();
    return b.statusesToTweets();
  }
  

  This mapping returns all collected tweets by calling completing a search by calling the handleSearch() method in the Bot object then returning the ArrayList<Tweet> returned by statusesToTweets() as discussed earlier. This returns a JSON structured array of Tweet data.

• /getRecent mapping. Returns only the most recent tweet in JSON format.

  @GetMapping("/getRecent")
  public Tweet getRecent() {
    b.handleSearch();
    return b.mostRecentStatusToTweet();
  }
  

  This mapping returns one JSON structured tweet object which is the most recent tweet found, returned by the mostRecentStatusToTweet() function as discussed earlier.

Deployment

Twitter Face Filter Bot Overview

Basic Overview

The Twitter Face Filter Bot is a project I decided to undertake in the summer of code program hosted by Mohamed Amine Belabbes during the summer between my first and second year of University in 2021. Its main purpose is to retrieve media from a tweet which mentions the page shown below. When media is retrieved the bot then uses machine learning to place a filter over the face of people in the image currently it places a batman mask over the face of the person in the image. It has two major components, a REST API System, written in Java using Twitter4J and SpringBoot libraries and a Machine Learning System used to apply the filter to the image, written in python using OpenCV and MediaPipe.

Example of Bot's Work

1. Tweet mentioning bot is posted on twitter.

2. Bot REST API System returns most recent tweet including ‘@FaceFilterBot’ in JSON format. When the bot makes a GET request to the REST API.

3. Bot then retrieves the media in the tweet and applies the filter to it using OpenCV and MediaPipe libraries in Python. If the media is a Photo it simply applies the filter to it, using the machine learning methods that will be discussed further in the post on a later Bot Filter System Post. [Placeholder Temp]