Resources for Communication Studies majors focusing on journalism

The new frontier of textual analysis - web and social media scraping

Web scraping is harvesting data from websites through either an API or a page's web browser. The point of web scraping is to collect an enormous amount of information relatively easily and quickly. It works by:

  • Downloading a web page's content
  • Extracting that information en masse 

Twitter Scraping Tools

Other Scraping Tools

Text Analysis Tools

Other (free) Sources of Data


Tweepy Tutorial

Python, Data Science, Text Analytics blog by Marco Bonzanini

Beautiful Soup Guides

Twitter scraper tutorial with Python by Randy Daw-Ran Liou

Beginner's Guide to Collecting Twitter data by Allen Zen

Guides from Twitter

Developer Docs

Webscraping Guide

Research Guide from Temple University