Crowdsourcing Bib - RGC Grant

From metadata-mediawiki.slis.ua.edu
Jump to navigation Jump to search

(Temporary spot until account is established on other wiki)

Project and Platform Design

Directly addresses crowdsourcing within digital humanities. Contains a solid lit review covering major publications to date with a focus on defining crowdsourcing for digital humanities and making recommendations for best practices.

Report on the Australian Newspaper Digitisation Project – a foundational case study on the design, management, and assessment of crowdsourcing in GLAMs. Provides an excellent model for employing usability testing to gauge the feasibility of a crowdsourced project. Furthermore, it presents a concrete model for assessing participant motivation through questionnaires distributed to the most active users. Finally, this study demonstrates how to measure the quality of crowdsourced transcription using confidence levels.

Outlines tips for designing and managing crowdsourcing platforms that maximize the benefits for libraries. These include: having one main transparent and clear goal on the home page and a visible charting of progress toward that goal; making the overall environment easy to use, intuitive, quick and reliable and the activities easy and fun; keeping the site active by addition of new content/work; giving volunteers options and choices; and rewarding high achievers with rankings that encourage competition. Illustrates these tips using screenshots from successful projects.

Identifies two main challenges: recruiting and maintaining knowledgeable users over time and maintaining the quality of user-generated metadata. Concludes that crowdsourcing volunteers are primarily motivated by connectedness/membership and sharing/generosity, with secondary motivations of altruism, fun, and competition. Also, crowdsourcing design must build in sufficient quality control and verification safeguards.


Research Study

Transcribing Handwritten Records

Data Extraction and Using Spreadsheets

Transcription Tools

Methods Articles

Provides an overview of the LIS literature on crowdsourcing and describes a number of crowdsourced cultural heritage projects and describes the “revolutionary cataloging interface” provided by Operation War Diary (OWD) – a major crowdsourcing transcription project administered by the UK’s National Archives and The Imperial War Museum. Includes detailed discussion of a usability testing survey administered to measure the OWD tutorial and interface and quality control mechanisms built into this crowdsourcing model.

Report around five key questions: is the crowdsourcing of manuscript transcription cost effective; is it exploitative; is the product high quality; is the project sustainable and the product permanent; and how can success be measured. Provides a review of major crowdsourcing projects to extract data, correct OCR, and transcribe indexes while noting that the transcription of manuscript collections is more complicated. The meat of the article addresses conceptualization and execution of the project including: development of the transcription tool (a customized MediaWiki allowing for encoding of linguistic and bibliographic data using TEI XML tags); the contributions of transcribers, which were significant; moderation by staff, which was intensive; and assessment of impact and success. Describes user survey employed to gauge factors that either motivated or discouraged volunteers.


Feasibility Study


Background Readings

According to this article, what we are doing can be characterized as a form of "content crowdsourcing" (CS) that they refer to as "Integrative Crowdsourcing":

Integrative CS will be relevant when the client firm seeks to build data or information bases. Therefore Integrative CS is a form of content Crowdsourcing. While gathering information or data at an individual’s level can be unproblematic, building a data base generally requires significant amounts of resources. The rationale of integrative CS therefore lies in the cost of building large data or information bases. Since individuals within the crowd are heterogeneous, Crowdsourcing enables the client firm to gather a variety of contents. The firm seeking to implement integrative CS should however be aware of integration issues. Data or information stemming from various origins might be incompatible or redundant if no precaution is taken. Precautions include the definition of a data format and the sound selection of data sources. [Emphasis added]

There are other relevant sections of this paper including:

  1. A way to distinguish 1992 data transcription tasks and the more complex cognitive efforts required for data extraction from the written articles from the 1961 season. See these sections of the paper:
    1. Crowdsourcing of simple tasks
    2. Crowdsourcing of complex tasks
  2. Discussion of incentives in this section: Discussion: benefits and pitfalls

This article has a great Guide to Crowdsourcing!!! (it's a subsection near the end of the article)

This should be one of our "go to" resources. It's a short book. This quote is indicative of its usefulness:

Crowdsourcing is a problem-solving model because it enables an organization confronted with a problem and desiring a goal state to scale up the task environment dra-matically and enlarge the solver base by opening up the problem to an online community through the Internet.

This quote is from Chapter 1 of the book

  • Terras, M.M. (2016). Crowdsourcing in the Digital Humanities In Schreibman, S., Siemens, R., and Unsworth, J. (eds), (2016) "A New Companion to Digital Humanities", (p. 420 –439). Wiley-Blackwell.

This chapter has a section that is a nice background discussion of historical document transcription.

Retrieved from "https://metadata-mediawiki.slis.ua.edu/index.php?title=Crowdsourcing_Bib_-_RGC_Grant&oldid=581"