Digital preservation in the age of AI and information overload

Digital Preservation in the Age of AI and Information Overload

New York City, USA – October 2023: Peter Hall, a computer science graduate student at New York University’s Courant Institute of Mathematical Sciences, has raised significant concerns about the future of data preservation in an age increasingly influenced by artificial intelligence (AI). Hall’s observations highlight issues around the permanence and fidelity of digital data, as well as the vital resources needed to maintain an accurate digital archive.

For many, the digital age has been marked by a paradox. While the internet is often viewed as an eternal repository, several users have discovered that the opposite can be true. Families have lost cherished photos posted on now-inaccessible social media accounts, favourite shows have disappeared from streaming platforms, and creators of all kinds have witnessed the erasure of their works with the shutdown of various web companies and platforms.

Simultaneously, AI tools such as ChatGPT and Midjourney have risen to prominence, promising to take over tasks traditionally performed by humans, from writing to video creation. This has led to predictions of a deluge of AI-generated content that could overwhelm and overshadow human contributions. Hall underscores that this influx raises significant privacy and data integrity issues, which should concern not just computer scientists but everyone.

Hall points out that data preservation ultimately boils down to resource allocation: Who will bear the responsibility of storing and maintaining information? Additionally, who will bear the financial burden? The developers of foundational AI models are among the key stakeholders in the quest to catalogue online data. However, their priorities do not necessarily align with those of the general public.

The costs associated with maintaining extensive data archives are considerable. Servers and the electricity required to keep them operational are not inexpensive, and these costs grow over time. Much like physical infrastructure, digital infrastructure requires ongoing maintenance. Hall explains that small-scale content publishers often find these costs prohibitive. Furthermore, simply backing up the internet periodically is insufficient without a mindful and structured approach to archiving.

Compatibility is another issue. As technology evolves, the formats in which we save our data today may become obsolete. Future preservation efforts might necessitate maintaining older computers and software to ensure continued access.

The question of intellectual property further complicates matters. Preserving data must be done in accordance with copyright laws to avoid litigation. Platforms like Spotify, which spent over $9 billion on music licensing last year, illustrate the potential financial stakes. This challenge is exacerbated when works have multiple creators or have changed hands over time, complicating permissions for preservation.

Hall asserts that in the internet age, distinguishing true and useful information from false or irrelevant data is more challenging than ever. Previously, the high cost of producing physical media naturally limited the dissemination of information. Online, the barriers are much lower, enabling the spread of misinformation. Effective data preservation must also address the quality and veracity of the information being saved.

The rise of AI-generated content introduces additional complications. Tools like ChatGPT are criticised for memorising training data, hallucinating false information, and potentially offending human sensibilities. As AI-generated content proliferates online, ensuring the preservation of valuable data becomes more difficult.

Hall suggests that AI-generated data, given its reproducibility, does not necessarily need extensive preservation efforts. However, he notes that even AI developers are wary of the impact of low-quality, synthetic data on their models.

The role of government in addressing these challenges is pivotal, Hall argues. Government institutions have the resources and the legal authority to support large-scale data preservation efforts. Libraries, for example, already preserve a vast array of physical media. The Library of Congress maintains some digital archives, focusing on historical and cultural documents, but this is far from comprehensive.

Groups like the Wikimedia Foundation and the Internet Archive contribute significantly to data preservation, despite facing financial and legal challenges. These organisations rely heavily on donations and volunteer efforts. The Internet Archive, in particular, has encountered legal obstacles related to copyright issues, potentially threatening its scope and viability.

Hall advocates for an expanded role for the government, particularly in empowering entities like the Library of Congress to archive digital data more comprehensively. He suggests that with sufficient funding, the government could overcome the legal and financial hurdles associated with data preservation.

Ultimately, Hall emphasises that data archiving should not end with government intervention but views it as a critical first step. He warns that neglecting to modernise archival practices could jeopardise the preservation of collective knowledge as libraries, at least in New York City, face funding cuts and closures.

In summary, Peter Hall’s insights highlight the complexities and challenges of data preservation in an era shaped by rapid technological advancements and an ever-growing sea of digital content. The necessity to devise a robust, resource-backed strategy for archiving valuable information has never been more urgent, given the precarious balance between resource allocation, legal considerations, and the relentless march of technology.

Source: Noah Wire Services