Hey Reddit! We are the folks behind the Internet Archive. We are really good at saving lots of stuff and putting it online forever, for free. We're trying to create the Web we want. Ask us anything!

First of all, I'd like to thank you for all of the work that you guys do. My brother and I are on the second year of what we call "Public Domain Halloween" and view a different horror movie each week in October that we find on Archive.org. It's been a lot of fun and I appreciate what you all do.

I'm a big film fan and have really enjoyed looking through your site for long-lost films that don't get enough love anymore. Are there any films that are on the site that you suggest/think that more people should enjoy?

"Neighbors" by Normal McLaren is a pretty amazing Oscar and Canadian Film Award-winning stop-motion film from 1952. Picasso called it the greatest film ever made.

https://archive.org/details/neighbours_20170314

-- Jason Scott

Could each of you pick an item in the collections that you find particularly interesting or meaningful and tell us about it?

I curated this Collection of books written in the 20s and 30s by Leo Edwards: r/https://archive.org/details/leoedwards  I had read many of them in my youth and wanted to make them available to young people today.  So I purchased them on ebay, digitized them, and now they are available forever for free.  

- Mark

In the home movies collection, there is a rare recording of a Japanese Internment camp from 1944.

At an Internet Archive event, one of the residents of this camp came and told his story of being a young boy at this camp and what his experience was. The debate in the comments below the film highlight how the US has not yet
come to terms with what these were.

-- Jason Scott

I am particularly moved by this visual rendering of the poem The Hangman by Maurice Ogden:
https://archive.org/details/the_hangman_1964

It was written in 1951, during the era of McCarthyism, and it admonishes us to take action against injustice when we see it, not just when it affects us directly. I think that message remains important today.

--Alexis

I've managed to consistently lose all of my data over several decades, yet you guys managed to preserve much of my early web portfolio that would otherwise be lost. What have you got to say for yourselves?

You're welcome? :-)

-- Alexis

What is it that you do on a typical day at your job?

My job description is a little floaty, so I try to go where needed, but my current (October 2018) duties include:

• Tech support around the Emularity, the Internet Archive emulation system that does all those neat software-in-browser things like arcade games, console games, home computers, handhelds. User contributions and ingestion of new sets of materials take a little bit of hand-holding to get right. (Right now, I'm screenshotting 35,000 Commodore 64 floppies, that's taking a bit of my attention)

• Sorting and cleanup of collections on the archive, either re-filing items that are in too-general buckets or contacting people who have uploaded 500 of something to see if they'd be fine with being given a collection of their own. Sometimes people are uploading dozens of computer manuals, for example, and that gets put over into a computer manuals section.

• Handling a lot of mass contributions via "take it from my directory" or providing a drop-off so people who have one big contribution (or sets of big contributions) don't need to learn every in and out. (For example, some people frequently hand me scanned/imaged CD-ROMs and I can help with that).

• Answer a lot of mail asking about operation, projects, and possible multi-faceted projects with the archive.

• Scan the social media to "help" when someone asks about the Archive in various ways.

-- Jason Scott

Typical?  Wow…. not sure we have had one of those recently.

There are so many different dimensions of the Internet Archive.  Starting with a focus on the various media we work to collect, preserve and make available.

The Web, Social Media, Books, Software, 78s, TV News, etc.

The range of tasks crosses the spectrum from the technical (software development), to operations (we host our own servers) to the domain specific (exploring with someone how to best collect and preserve a specific collection or media.)

My primary responsibility is to lead the efforts of the Wayback machine to archive as much of the public web as possible.  To help serve the mission statement of the Wayback Machine to “help make the web more useful and reliable.”  I work with a team a engineers (paid and volunteer) around the world who are developing and operating technology to help do that.  At the same time I work directly with various platforms and sources of content (e.g. Wikipedia, Wordpress, news orgs) to ensure we do the best job we can to archive what they make available. 

I think the most important part of my job is actively asking how we can do better, listing to what people tell us, and helping to be responsive to that input.  

So… how can we do a better job of helping to advance the mission of “Universal Access to all Knowledge”?

- Mark

I tend to browse the archives and the Wayback Machine for endless entertainment, but I'd love to hear about a time where the Wayback Machine and/or other information from your archives has served a greater purpose? I can imagine they have been integral for particular discussions around things such as "fake news" and bias online as of late...

If you do a search for "Internet Archive" and "Wayback" on most news search engines, you will find many news organizations have used us for comparing deleted or changed web pages for the purposes of reporting. There are cases where the Wayback has been used without credit/using our name so there are lots of additional examples as well.

I've had numerous e-mails from teachers and educators who have used the Archive's software emulation and digitized magazines as class assignments to allow students to do research for homework and reports.

​

One particularly touching e-mail came from a musician whose previous band had a partner who died, and whose family did not approve of the band, and had destroyed all the music in his possession. Our mirror had all the music that was lost, and so this musician could re-live the better times with his friend and collaborator. This sort of story is repeated a lot.

-- Jason Scott

What is something that each of you is passionate about archiving?

In general, I am passionate about what I call "Non-Advocated Material", which some call "Ephemera" and others might call "Disposable" - items which could never hope to get funding, or support from an institution, and which passionate outsiders or volunteers insist on making available digitally for the world. There's little question that presidential papers, or court cases, or the founding documents of an institution will be preserved (or at least, accepted as candidates) but I breathe a sign of relief when I see people have done some digitizing of:

• Pogs
• Manuals for the AAMBER Pegasus, a computer that sold less than 100 units
• Atari 800 Program Source Code (Digital and Printed)
• Rave Zines
• Restaurant Menus
• Fakebooks

When I see someone has taken item to scanner without worrying about the "usefulness" or "meaning" or "where will we get the grant", I always think angels walk among us.

-- Jason Scott

I am especially passionate about this archive of web content from and about North Korea: https://archive-it.org/collections/6777

- Mark

I am particularly interested in archiving media by/about/for traditionally underrepresented communities. One of my personal favorites is the Dorothy Fadiman archive at https://archive.org/details/fadimandocumentaries which contains documentaries from an Oscar winning film maker about social issues including education, women's rights, HIV prevention, and abortion. -- Alexis

​

First, thank you all for your amazing work.

Reading through this thread I realized that some of the tasks you perform must become extremely monotonous. In particular this comment from Jason spurred my question:

Right now, I'm screenshotting 35,000 Commodore 64 floppies, that's taking a bit of my attention

How do you break the monotony, when needed, do you tend to jump between projects?

Also, I'm curious to know what projects you enjoy the most, not necessarily the ones you find most meaningful.

To clarify, I'm not playing 35,000 floppies, ctrl-C and ctrl-V, repeat - I have a machine that is playing these items, making screenshot sets, and then doing a number of analysis on them (did it work, are there duplicates, etc.) and then uploading them, taking about 2 minutes per item. Another script is dealing with the situation that Commodore 64 (specifically) has a very dull blue READY screen that gets replaced with a much more interesting visual when the program runs, and this script will make that the "icon" image. My role in this is to occasionally monitor the incoming screenshots, see if everything is flowing, and track down when my scripts do things they're not supposed to. So, boring, but not night-security-job boring.

I have, on average, 20-30 windows open to the archive via terminal windows as well as 10-15 webpages that I'm looking at. I find the integration and addition of information to be soothing and enjoyable - I am rarely bored. If a task requires me to type or analyze (i.e. type summaries or add metadata) then I listen to music, and if it is purely visual or click-based (I have a bunch of those) then I play podcasts.

-- Jason Scott

I love our Television News Archive. We have been archiving about 50 TV News channels 24/7 for the past 9 years and make much of that video available, supported by full-text search based on both the closed captions and the text in the lower-third chyrons.

In encourage you check out the "Visualizations and Press" section of: https://archive.org/details/tv to learn more about this project

- Mark

I find this collection of vintage TV ads endlessly entertaining, and since they're so bite sized they can be a quick little distraction as well:
https://archive.org/details/adviews

Also, I like listening to the old Jazz records from the Great 78 Project while I'm doing anything tedious:
https://archive.org/details/georgeblood

I also break things up by finding fun stuff to post on our social media accounts on Twitter and Facebook.

-- Alexis

What file systems and operating systems are your servers using? Tell us some impressive specs of them!

We're currently running Ubuntu Linux (primarily Xenial Xerus), and with a few exceptions our underlying storage media resides on ext4 filesystems, with redundancy/recovery logic in higher layers. As an archive, we tend to err on the conservative side. We have more than 40 petabytes of unique data, all of which is stored at least twice. We sustain about 35 to 40 Gbps of outbound traffic.

-- Alexis

Recently I used the wayback machine to find an old user manual on a now debunked wepage, but my question is do you usually try to archive files and documents like that or was I just lucky that someone else submitted it?

There are multiple ways to gather items into the Wayback machine - we have internal and external processes moving through the web and taking samples, and once a site is archived in the Wayback, we re-check that site later in time, automatically. There is also a "Save Page Now!" function on the web.archive.org page which many folks use when they feel something is of notability or likely-to-change status, which catches some users off guard that the Wayback "knew" to archive a page - no, we just had really good users point the machine at the resource.

The web is very very big, so there's always a chance that a website will be missed, especially if it has few external links to bring it to attention - but in general, the wayback and partner organizations are doing an awful lot of crawling, an awful lot of the time.

-- Jason Scott

Both! I think you were lucky and we do go out of our way to archive user manuals and other information that might be considered "important" (to someone) and potentially "at risk". Other categories of content that might fit that bill include information about climate change, government programs (all governments world-wide), political campaigns and politics in general and scientific/academic works. As you can see we cast our nets wide and attempt to go deep at the same time. It's a big web, someone has to back it up!

- Mark

Are you guys involved in litigation from time to time? Like if someone wants to prove they put content up before someone else.

Yes. Content from the Wayback Machine is often presented as evidence in legal cases and various courts have ruled on the admissibility of that evidence. For example see: https://www.theregister.co.uk/2018/09/04/wayback_machine_legit/

https://www.law.com/newyorklawjournal/2018/07/16/the-wayback-machine-in-the-courts-introducing-evidence-of-the-past-state-of-the-web/

- Mark

Thanks for asking all these great questions! We're out of time, but you can always find us in the archive. :-)

-- Alexis, Mark, & Jason

Do you have a revenue model or do you rely on contributions?

​

We bring in funds in several different ways. Here are the biggies:

- Digitize books for other libraries and institutions

- Archive the web for people (see archive-it.org for example)

- Donations from our awesome supporters at https://archive.org/donate/

- Foundation support through grants and donations

​

So yes! Donations are incredible helpful! But we also have other revenue sources to rely on.

--Alexis

Hi Internet Archive crew! Thanks for all that you do. Glad to have you here doing an AMA.

I'd love to hear what you think is the single-most important thing individuals can do to protect Internet freedoms? Given so much is at stake these days, what would you tell someone looking to get involved and defend free and open access to information online?

​

Personally I think there are two important paths here - Net Neutrality and Decentralization.

Net Neutrality protects our access to the "pipes" themselves, allowing us to communicate and consume media that we think is important without artificial slow downs or preferential treatment. Learn more here: https://www.eff.org/issues/net-neutrality

Decentralization could help us by putting control of our data and information back into users' hands. This might be a helpful overview if you want to learn more: https://www.computing.co.uk/ctg/news/3036546/decentralising-the-web-the-key-takeaways

-- Alexis

Alexis just gave a much more substantive answer. But... I can tell you what I do. I am a long term member, and supporter, of the Electronic Frontier Foundation. Eff.org

- Mark

Off the top of your head, what's the weirdest thing archived?

I could answer this question ALL DAY LONG!

Some personal favorites:

https://archive.org/details/electric_eel (Scientist electrocuting people with electric eels)

https://archive.org/details/MortenSvinthJensenBlottilLyst_0 (because puppets are awesome)

https://archive.org/details/The_Mascot_Complete (because puppets are terrifying)

https://archive.org/details/journey_to_the_center_of_a_triangle (because I didn't know math was so soothing)

-- Alexis

​

​

None of these links disappointed. I don't know if I can finish the 3rd video so close to Halloween though.

OMG I FORGOT SALAD FINGERS!

https://archive.org/details/SaladFingersFullSeries110

-- Alexis

When I first joined the Archive in 2011, I asked them to scan in a braille edition of Playboy magazine as a stunt.

Everyone at the Archive wondered what they heck they'd hired, but it got attention!

-- Jason Scott

What is the most challenging aspect of the work you do?

Like a lot of great things in the world, we have folks who know all about us and all the different projects and resources we have... and then we'll get in a news story or mentioned somewhere prominent and there's way too many comments of "What? Where has this place been all my life?".

We keep things spinning and our projects humming 24/7, so any time we're not known about by someone is a missed opportunity. Reaching these potentially benefited audience members, future patrons, without sounding like a broken record to whoever already knows about us, is a challenge.

-- Jason Scott

Backing up as much of the public web as possible presents a slew of challenges.  First off the Web is "big and messy".  And growing more so every day.

Trying to preserve web-based resources so they can be “replayed” in the future is becoming more difficult as they are increasingly personalized and dependent on back-end databases and 3rd party APIs.  On top of that, various media formats are deprecated over time.

Add to that the fact that a growing number of Web-based resources are behind fire and pay walls, and otherwise not generally available to our crawlers and other archival tools.

Additionally, the volume of material humans create far exceeds our ability to “get it all”.  As such the process by which we prioritize what we think may be most important to archive for future generations becomes more of a challenge. 

- Mark

How is the Openlibrary project going?

Open Library (openlibrary.org) is our most active community of volunteer developers, led by Mek Karapeles. They have been making great changes to the site over the last couple of years to make OL better for users. If you'd like to help out, check out the documentation at https://openlibrary.org/developers

My favorite new feature this year lets you track the books you're reading. Check it out! http://blog.openlibrary.org/2018/03/07/share-what-youre-reading/

-- Alexis

Thanks for all your work! How much impact does your work have? Can you point to groups that are using your website to change society?

The Wayback Machine or IA's other resources (like the Television News Archive) have been used in about 1,800 news articles over the past 2 years alone. See: https://archive.org/about/news-stories/search/

Also, we are used by phd students to write their theses, print disabled people who can now access a wider array of books, people who are archiving documents from FOIA requests, archives of other types of government documents like Public Resource (https://public.resource.org/) and Recap (https://free.law/recap/), among many others.

-- Alexis

Hey Guys,

Use your service often, thank you. I want to create my own archive of a few different websites. Is there a commercial service you can recommend? HTTrack is close, but not quite what I'm looking for. I'm looking for something that I can put in the URL and a time and it will automatically download on that time. And simple, because I'm a tech idiot.

Appreciate any recommendations.

The Internet Archive offers a paid subscription service called Archive-It. You can learn more about it here: https://www.archive-it.org

Do you archive complete sites including back end scripts and database? Or just rendered static pages?

The wayback machine does not archive anything you can't see from the front-end. That said, there have been rare cases of sites packing all their assets up, throwing them into a .tar or .zip file, and uploading that as a final goodbye. Obviously these don't end up in the wayback machine, but the thought is appreciated.

-- Jason Scott

If someone or some company contacts archive.org to remove something from the waybackmachine, what's the process like?

Generally speaking we respect site owners’ requests for archives from their site to be removed from the Wayback Machine.

Having said that, at times, special consideration is given to the public’s interest for access to material, and that interest might override a site owners request.

- Mark

Hello and thank you for the invaluable work of archiving!! Do u accept old games to emulate them on your site??

We have been sent many games from original authors or sets of games from sites that are running low on hosting/disk budget and need them to stay alive.

-- Jason Scott

How do you feel about websites banning archiving of their domain?

Disappointed.

-- Jason Scott

In general terms I don't like it. Especially when the site in question is a news or government site. However I can appreciate cases were certain sensitive content may be made available on sites that are considered ephemeral and where the long-term (forever) access to that information may put some people at risk.

- Mark

What is the very best cheese?

"Crap from the Past", a show of pop music cheese that has been running for over 16 years and which the creator/DJ has been curating all that time, including mirroring on the archive. https://archive.org/details/crapfromthepast&tab=about

-- Jason Scott

This cheese:
https://archive.org/details/SayChees2001

--Alexis

I am fond of: https://en.wikipedia.org/wiki/Reblochon

- Mark

Until recently, I wasn't aware Archive hosts such a good library of books. However, it's mostly older stuff, released in early 00s at best. Are you planning to add newer books in (possíbly even far) future? And how it's affected by copyrights? Does the author (or his successors) have to personally agree you can share the book, for two weeks, but still for free?

I'm glad you're loving the books! We have 800,000 books (including fairly new ones) available through our controlled digital lending project at https://archive.org/details/inlibrary --Alexis

For more information about Controlled Digital Lending see: https://controlleddigitallending.org

- Mark

I've uploaded to the LMA (live music archive) in years past and really appreciate the free hosting and converting to various audio formats. What's something you like from the LMA and how many folks help run it?

What is the most challenging aspect of the work you do?

I also like the fact that all the music from the Live Music Archive (as well as our collection of digitized 78s (100,000 and counting)) is now available via Google Assistant devices. Try "Hey Google, Ask Internet Archive" Or, "Hey Google, Ask the Internet Archive to randomly play the Grateful Dead" or "Hey Google, Ask the Internet Archive to randomly play Jazz 78s".

- Mark

LMA is run by an amazing and active group of volunteers from http://bt.etree.org/. They do all of the heavy lifting, and we just provide a platform for storage and file hosting and give them an occasional assist with code changes.

I like the Ditty Bops! https://archive.org/details/TheDittyBops

--Alexis

I like chiptunes, so "I Fight Dragons" gets my vote.

https://archive.org/details/ifd2009-10-14.como-flac16

-- Jason Scott

What do you think about people using thewaybackmachine to find old posts to repost on reddit?

I think it all depends on the context. If people are able to use the Wayback Machine to help set the record straight or otherwise help us remember, and learn from, our past, it is a good thing. To the degree the Wayback Machine helps contribute to the constructive and civil exchange of ideas, and supports healthy discourse, we are happy.

- Mark

Do you keep alive any domain names with its content?

To take this further: It has been extremely rare for the Internet Archive to 'take over' operation of anything like a domain or business or online experience, in a way that would be "it's still running, but now the Archive runs it". Running a website, even a static one, takes administration and development resources that we simply don't have - keeping the archive.org site/projects going and improving them is where our dollars go.

One thing we have done is be the source of a redirection - for example, the Internet Underground Music Archive (IUMA) was mirrored from an external backup and is hosted here. This delighted a few of the (long-unaffiliated) founders and one redirected the IUMA.COM domain to our collection, so for years, you would go to IUMA.COM and it'd end up at the Archive. But checking today while answering this question, I see that this no longer works - a perfect demonstration of how even the relatively simple process of keeping a domain that just redirects all requests may fall out of usage or maintenance and disappear.

Generally, we have mirrors of sites in the Wayback or, if a site had content or software that it was distributing that was part of the ecosystem at the time, a mirror of that might show up as an item in our collections. Anything more intense a "takeover" is generally much more trouble than it is worth.

-- Jason Scott

We have been archiving much of the public web for the past 22 years and make those archives (341 billion web pages and growing) available via the Wayback Machine.

Many billions of those pages are no longer available via the "live web" and are now only available via the Wayback Machine.

E.g. there are more than 9 million URLs, from 26 Wikipedia Language Editions. that had gone bad (returned a 404) that have been edited to point to Wayback Machine URLs.

- Mark

Do you algorithmically delete/purge archives? I

No. --Alexis

What's a collection or item that people should go experience in person, that's not well served by digital redistribution? (yet!)

The Internet Archive is a non-profit, as such we don't get stock options. However one way we honor the service of the staff is that when people work here for more than 3 years a ceramic statue is made of them. They are located in the "Great Room" of our office (an old Church) in San Francisco. You can learn more about them, and our office, here: https://richmondsfblog.com/2014/04/29/internet-archive-installs-100th-archivist-statue-will-rebuild-building-lost-in-fire/

- Mark

Will you ever update the UI/logo?

You don't like retro? --Alexis

[deleted]

We did actually completely redesign the site in 2014. You can read about it here (as well as see some screenshots of previous versions): http://blog.archive.org/2014/11/05/redesign/

[deleted]

We really like that logo, and look what happened with Pepsi.

-- Jason Scott

Maybe. Are you, or someone you know, a designer? We might not pay well (or at all!) but it would be hard to argue that the right person, with the right ideas, would have an opportunity to have a positive and meaningful impact on millions of Internet Archive patrons for generations to come. Note, I am speaking more about the overall UX/UI and less about the logo.

However, if we did update the logo we might have to find a new building. Little known fact, one of the reasons the Internet Archive purchased our office (Church @ 300 Funston, SF) is because the building matched our logo.

- Mark

Are you hiring librarians/archivists who can work remotely from Chicago? Asking for a friend who is actually me

Our jobs listing is usually the best place to check: https://archive.org/about/jobs.php

-- Jason Scott

Hi. I read the IA has a "Book Drive" where people can donate their books, so it can be scanned and preserved for ever. Is the Book Drive still running, how long (aprox) does it takes for a book since it arrived to you to be scanned and shared on the OpenLibrary ? Regards

We are still accepting book donations for titles we don't have. You can check our collection by searching for the book on https://openlibrary.org/ - anything with an ebook or DAISY file (for print disabled patrons) has already been digitized.

We digitize about 1,000 books per day at the moment. We do not have an estimated time from donation to digitization, though.

-- Alexis

Hey, how come you can't edit the category of an item you uploaded after you've archived it? Sort of a small thing but I just wanna know.

Do you mean the collection? Generally, most uploaders don't have privileges to upload into anything but one of the Community collections.

-- Alexis

Hi. Beside uploading content to the IA and OpenLibrary, what other activities an "Internet Archive Online Volunteer/Community Member" can do to help the library? ... beside donating more money :)

Open Library has a lot of active volunteers who do coding and other work. We also have opportunities for volunteers at local events as well. You can find more about that here: https://archive.org/about/volunteerpositions.php

-- Alexis

Do you guys sponsor visa for international workers? I really admire what your guys have done for the Internet but I cannot work in the US without a proper visa

All I can do is direct you to our Jobs page, which lists jobs we're looking for, and how to contact us about them. I'd make a visa question the first part of your letter.

https://archive.org/about/jobs.php

​

- Jason Scott

The real questions: roughly how much porn have you archived... total or this week?

Probably more than enough. :-)

-- Alexis

How do I remove my website and all history from your service?

Please send email to [[email protected]](mailto:[email protected])

- Mark

How is your funding doing?

Great! But we can always do better. One of the ways we accept support is via direct contributions from people who come to our site. In a few weeks we will kick off our "end-of-year" fundraising effort and invite you to help support our work. Last year about 73,000 people told us how much they appreciate what we do by contributing to our end-of-year campaign. If anyone would like to explore how else they might help support our work please reach out to us via [[email protected]](mailto:[email protected])

- Mark