To learn more about who links to Wikipedia, our research team at the University of Sheffield analyzed the structure of links that point to Wikipedia pages from external websites, looking specifically at which top-level domains dominate the link volume for each Wikipedia language. Here’s what we found.
Key findings
- The most popular Wikipedia according to the number of links that it receives from external websites is the English Wikipedia. After that, the Spanish, the Indonesian, and the German Wikipedias are the most popular.
- Websites under blogspot.com contribute many links to Wikipedia. For the Spanish, Indonesian, and Portuguese Wikipedias they contribute more than all the other .com websites.
- The main top-level domains that link to the English Wikipedia are: .com, .org, .blogspot.com, .net, .edu, and .co.uk.
- Out of the 36 million links to Wikipedia, 18 million links are from .com domains to the English Wikipedia.
- Most links to the German and French Wikipedias come from .de and .fr domains respectively.
Data
In early 2015 the website WikiReverse.org published raw data about all the 36 million hyperlinks from any external website to any Wikipedia page. Such data was extracted from CommonCrawl (read more about the extraction process on the WikiReverse website): a large crawl of the Web run by the Common Crawl Foundation in July 2014.
We have analyzed the WikiReverse dataset to understand the linking structure from the external Web to Wikipedia. We have aggregated and visualized this data according to the top-level domain (e.g., .com, .net.) of the websites linking to Wikipedia. The list of top-level domains we aggregated the raw data against has been manually curated. We then visualized the aggregated data using Tree Maps to show the proportion of the link volume between domains and Wikipedia languages.
The result for all Wikipedia languages and all data is shown in the graph above. The first line of text in each box indicates the Wikipedia language. The second line indicates the top-level domain. The size of each box is proportional to the number of links from websites having a certain top-level domain to a certain Wikipedia language. Here you can find the interactive plot with the link volume for all Wikipedias
As the English Wikipedia dominates the visualization, this is the resulting visualization for all non-English Wikipedia languages:
Here are the top ‘non-english’ external sites that link to Wikipedia, excluding link volume for English-language top-level domains. Graph by Gianluca Demartini, CC-BY-SA 4.0.
Here you can find the interactive plot with the link volume for non-English Wikipedias
The code used to process the raw data from WikiReverse can be found in this iPython notebook.
Limitations
The original data used for this analysis and observation is the Common Crawl which is an incomplete crawl of the Web. This means that all observation are valid only based on the data within Common Crawl and may not be representative of the entire Web.
Gianluca Demartini, Lecturer in Data Science at the University of Sheffield. gianlucademartini.net
