Roland Schäfer. Processing and Querying Large Web Corpora with the COW14 Architecture. In Proceedings of Challenges in the Management of Large Corpora (CMLC-3) (IDS publication server). 28–34. [BibTeX]

In this paper, I present the COW14 tool chain, which comprises a web corpus creation tools called texrex, wrappers for existing linguistic annotation tools as well as an online query software called Colibri2 . By detailed descriptions of the implementation and systematic evaluations of the performance of the software on different types of systems, I show that the COW14 architecture is capable of handling the creation of corpora of up to at least 100 billion tokens. I also introduce our running demo system which currently serves corpora of up to roughly 20 billion tokens in Dutch, Englisch, French, German, Spanish, and Swedish.

Corresponding talk at Challenges in the Management of Large Corpora (CMLC-3) on July 20, 2015 in Lancaster.