Crawls performed by Internet Archive on behalf of the National Library of Australia. This data is currently not publicly accessible.
106M
106M
Dec 19, 2017
12/17
by
Internet Archive Web Group
A series of open web crawls targeting journal articles, technical memos, essays, datasets, and other research publications. This collection contains WARC and CDX files that end up in Wayback ( https://web.archive.org ). See also bibliographic metadata corpuses at https://archive.org/details/ia_biblio_metadata
WARCs from internal crawl testing.
Topics: web, cctld
This collection contains collaborative Election crawls performed by IA.
Topics: elections, web
This crawl was performed in Summer & Fall of 2012 to archive the US Federal Elections.
Topics: US, federal, elections, web, 2012
Data collected by Internet Archive on behalf of the National Library of Spain. This data is currently not publicly accessible.
National Archives and Records Administration crawl performed by Internet Archive. This data is currently not publicly accessible.
Crawls of the french domain space performed by Internet Archive on behalf of Bibliotheque Nationale de France. This data is currently not publicly accessible.
National Library of Luxembourg
Topic: Luxembourg
Data collected by Internet Archive on behalf of the National Library of Sweden. This data is currently not publicly accessible.
this data is currently not publicly accessible.
National Library of Austrailia crawl. This data is currently not publicly accessible.
this data is currently not publicly accessible.
Data collected by Internet Archive on behalf of the National Library of Israel. This data is currently not publicly accessible.
Topic: nlil
This collection includes all collaborative Olympic crawls performed by IA for the IIPC.
Topics: olympics, IIPC, web
Domain crawl of the Australian web domain (.au) performed by Internet Archive on behalf of the National Library of Australia in March-April, 2022.
Topic: crawldata
This crawl of online resources of the 114th US Congress was performed on behalf of The United States National Archives & Records Administration (NARA).
this data is currently not publicly accessible.
5.4M
5.4M
May 28, 2020
05/20
by
Internet Archive Web Group
12.2M
12M
Aug 4, 2017
08/17
by
Internet Archive Web Group
Microsoft Academic Graph public corpus (Feb 2016) PDF URLs, filtered to remove large sites (pubmed, citeseerx, arxiv) and already-crawled URLs.
Topics: papers, journals
These crawls were performed by IA on behalf of the IIPC in Summer 2012 during and prior to the 2012 Summer Olympics held in London, UK.
Topics: London, olympics, web, 2012, IIPC
15M
15M
Jul 17, 2018
07/18
by
Internet Archive Web Group
Web archive data from a crawl of open access PDF URLs provided by Unpaywall.
this data is currently not publicly accessible.
10.1M
10M
Jul 6, 2020
07/20
by
Internet Archive Web Group
This collection includes content harvested from the Web on behalf of the National Library & Archives New Zealand in February 2013.
Topics: web, domain
Crawls performed by the Internet Archive in 2017 on behalf of the National Library of Australia.
Topic: nla web 2017
017-2022-Summer domain crawl of the Luxembourg web domain (.lu) performed by Internet Archive June - July 2022 on behalf of the National Library of Luxembourg / Bibliothèque nationale de Luxembourg.
Topic: crawldata
These crawls of the .es domain were performed in 2011 on behalf of the National Library of Spain (BNE).
Topics: bne, spain, web, 2011
This crawl of the .au domain was performed on behalf of the National Library of Australia in of 2016.
Topics: nla, australia, web
this data is currently not publicly accessible.
ccTLD crawl for .br domain
Topics: br, web, 2018, cctld
Crawls performed by Internet Archive on behalf of the National Library of Ireland. This data is currently not publicly accessible.
This crawl of online resources of the 112th US Congress was performed in Fall of 2012 and early winter of 2013 on behalf of NARA.
Topics: nara, 112th, web
Domain crawl of the New Zealand web domain (.nz) performed by Internet Archive on behalf of the National Library of New Zealand in January-February, 2018.
Topics: web, nlnz, 2018
This crawl of the .au domain was performed on behalf of the National Library of Australia in of 2015.
Topics: nla, web, 2015
11.1M
11M
Apr 9, 2018
04/18
by
Internet Archive Web Group
WARCS from Worldwide Government Web (WGW) Domain Crawls
Topic: crawl data
Domain crawl of the Australian web domain (.au) performed by Internet Archive on behalf of the National Library of Australia in March-April, 2021.
Topic: crawldata
this data is currently not publicly accessible.
This crawl of online resources of the 115th US Congress was performed on behalf of The United States National Archives & Records
Topic: crawldata
Crawls performed by the Internet Archive in 2018 on behalf of the National Library of Australia.
Topics: nla, web, 2018
This crawl of the .au domain was performed on behalf of the National Library of Australia in of 2014.
Topics: nla, web, 2014
Topics: bne, spain, web, 2013
2015 crawl of museum websites listed in the IMLS Museum Universe Data File. More about the IMLS MUDF can be found at https://www.imls.gov/research-evaluation/data-collection/museum-universe-data-file
Topic: AIT
Domain crawl of the New Zealand web domain (.nz) performed by Internet Archive on behalf of the National Library of New Zealand in January-March, 2022.
Topic: crawldata
this data is currently not publicly accessible.
5.6M
5.6M
Apr 26, 2019
04/19
by
Internet Archive Web Group
This crawl of the .au domain was performed on behalf of the National Library of Australia in Spring of 2013.
Topics: nla, web, 2013
Crawls performed by the Internet Archive in 2019 on behalf of the National Library of Australia.
Topics: nla, web, 2019
Domain crawl of the New Zealand web domain (.nz) performed by Internet Archive on behalf of the National Library of New Zealand in January-March, 2021.
Topic: crawldata
2017 domain crawl for National Library of Ireland.
Topics: ireland, web
this data is currently not publicly accessible.
Crawls performed by the Internet Archive in 2020 on behalf of the National Library of Australia.
Topics: nla, web, 2020
5.3M
5.3M
Feb 15, 2019
02/19
by
Internet Archive Web Group
This crawl of the .es domain was performed in 2012 on behalf of the National Library of Spain (BNE).
Topics: bne, spain, web, 2012
Crawl 00001 of the IMLS Museum Universe Date File.
this data is currently not publicly accessible.
this data is currently not publicly accessible.
3.9M
3.9M
Mar 5, 2020
03/20
by
Internet Archive Web Group
Web archive data collected by Internet Archive from a domain harvest of the Ukrainian ccTLD.
Topic: crawldata
this data is currently not publicly accessible.
26.7M
27M
Oct 3, 2013
10/13
by
dominic@archive.org
This crawl of the .il domain was performed in 2013 on behalf of the National Library of Israel (NLIL).
Topics: nlil, israel, web, 2013
this data is currently not publicly accessible.
This crawl of the .nz domain was performed on behalf of the National Library of New Zealand in Spring of 2017.
Topics: nlnz, web, 2017
Domain crawl of the New Zealand web domain (.nz) performed by Internet Archive on behalf of the National Library of New Zealand in January-February, 2019.
Topics: web, nlnz, 2019
1.7M
1.7M
Oct 31, 2018
10/18
by
Internet Archive Web Group
Crawl of "upstream" URLs from CORE (core.ac.uk) metadata dump. Only a partial seedlist of files crawled.
This collection includes content harvested from the Web on behalf of the National Library & Archives New Zealand in January 2016.
Topics: new zealand, web, domain
Domain crawl of the Israel web domain (.il) performed by Internet Archive in October-November 2021 on behalf of the National Library of Israel.
Topic: crawldata
Crawls performed by the Internet Archive in 2020 on behalf of the National Library of Israel .
Topic: web
3.8M
3.8M
Jan 24, 2020
01/20
by
Internet Archive Web Group
Data collected by Internet Archive on behalf of Biblioteca Nazionale Centrale di Firenze. This data is currently not publicly accessible.
This crawl of the .il domain was performed in 2015 on behalf of the National Library of Israel (NLIL).
Topics: nlil, israel, web, 2015
This crawl was a domain scale harvest of .au performed for the National Library of Australia in 2010.
Topics: nla, web, 2010
This crawl of online resources of the 111th Congress of the United States was performed in Fall of 2010 and Winter of 2011 on behalf of NARA.
Topics: nara, 111th, congress, web
