COMP 475 - Search Engine Development

Classes | Home

---

Dr. Frank McCown
Science 206
MW 3:00 - 4:15 pm

Class links

• Syllabus
• Blackboard - For taking quizes
• Easel - All homework and projects are here

Textbooks

• Required text book: Google's PageRank and Beyond: The Science of Search Engine Rankings by Langville and Meyer
• Online text: An introduction to information retrieval by Manning, Raghavan, and Schütze
• Alternate book: An Introduction to Search Engines and Web Navigation by Levene

Lectures

• Introduction

Search Engines

• Web Search Engine Wikipedia article
• List of search engines
• Popularity: Market share of US searches in 2007
• A Comparison of Open Source Search Engines by Middleton and Baeza-Yates (2007) and rebuttal
• Nutch

Tomcat, Java, and Java servlets

• Java and C# Comparison
• Java Vectors and Hashtables
• Configuring & Using Apache Tomcat 6
• Servlet, JSP, and JDBC Tutorials

Web Chracterization

• Map of the Internet
• W3C Web Characterization Terminology & Definitions Sheet
• Trends in the Evolution of the Public Web: 1998-2002 by O'Neill, Lavoie, Bennett (2003)
• Graph Structure in the Web by Broder et al. (2000)
• The Indexable Web is more than 11.5 billion pages by Gulli and Signorini (2005)

Search Queries

• Web search query Wikipedia article
• Google Trends
• Searching the web: The public and their queries by Spink, Wolfram, Jansen, Saracevic (2000)
• AOL search terms
• History Repeats Itself: Repeat Queries in Yahoo’s Logs by Teevan, Adar, Jones, Potts (2006)

Web Crawling

• Web Crawler Wikipedia article
• Simple web crawler written in Java
• Heritrix web crawler
• Web Robots Pages - Robots Exclusion Protocol
• Site Design Impact on Robots by Smith and Nelson (2008)
• Mercator: A Scalable, Extensible Web Crawler by Heydon and Najork (1999)
• Crawling the Web by Pant, Srinivasan, and Menczer (2004)
• Sitemaps

Indexing

• Search engine indexing Wikipedia article
• Inverted Index Wikipedia article
• Ch 1: Boolean retrieval
• Ch 2: The term vocabulary and postings lists
• Porter Stemming Algorithm and various implementations

Ranking

• PageRank Wikipedia article
• The PageRank Citation Ranking: Bringing Order to the Web by Page, Brin, Motwani, Winograd (1998)
• HITS Algorithm Wikipedia article
• Authoritative Sources in a Hyperlinked Environment by Kleinberg (1999)

Projects

• Web Crawler
• Indexer
• Search Index

Other Search Engine Courses

• WWW Search Engines at Lehigh University
• Web Search Engines at New York University

---

Classes

Page last modified: