Apparatus and methods for annotation management

12013,841

17/307628

May 04, 2021

Apparatus and methods to manage annotation of a body of data comprising data files are provided. The apparatus and methods may include an annotation management system concurrently accessible over a network by annotators, data engineers, data scientists, and checkers. The annotation management system may include a label management module, an annotated data management module, a change management module, a navigation management module, an original document reference, a database management module, and an output management module. The apparatus and methods may include locking a data file while an annotator is applying labels but concurrently allowing others to view and manipulate the data file.

Bank of America Corporation (Charlotte, NC, US)

Bank of America Corporation (Charlotte, NC, US)

1. A method for managing concurrent annotation of a body of data comprising a plurality of data files, the method comprising: (a) storing, at a centralized server, the body of data; (b) receiving, at an annotation management engine, a user login from an annotator, said user login enabling the annotator to select an available data file from the plurality of data files; (c) authenticating the annotator; (d) displaying, on a monitor proximate to the annotator, the selected available data file and a graphical user interface; (e) receiving, at the annotation management engine, one or more saved labels applied by the annotator to the available data file; (f) locking, at the annotation management engine, the available data file, said locking transforming the available data file to an annotated data file and preventing a different annotator from applying one or more saved labels; (g) updating the body of data with the annotated data file; (h) generating, at the annotation management engine, a history of the annotated data file; (i) receiving, at an annotation management engine, a user login from a checker while the annotator is labeling the available data file, said user login enabling the checker to select the annotated data file and the available data file from the plurality of data files; (j) authenticating the checker; (k) unlocking, at the annotation management engine, the available data file, said unlocking allowing the checker to check and apply one or more labels to the available data file; (l) displaying, on a monitor proximate to the checker, the annotated data file, the available data file, the history, and a graphical user interface; (m) receiving confirmation from the checker that the labels are accurate; (n) storing the confirmation and the annotated data file as a checked data file; (o) updating the body of data with the checked data file; and (p) generating, at the annotation management engine, a history of the checked data file; wherein the history includes the available data file, the annotated data file, the one or more saved labels, an identity of the one or more annotators, and any changes made by the one or more annotators to the available data file, and wherein the annotation management engine, through an artificial intelligence/machine learning (“AI/ML”) algorithm automatically determines available data files of the plurality of data files.

2. The method of claim 1 wherein the graphical user interface comprises: a navigation and function taskbar; available labels; a historical data section; and the available data file; wherein the annotator and the checker can apply one or more of the available labels to the available data file.

3. The method of claim 2 wherein the available labels are color-coded.

4. The method of claim 1 wherein at least steps (b), (e), (g), (i), (m), and (o) are performed over a network.

5. The method of claim 4 wherein the network is an internal intranet.

6. The method of claim 4 wherein the network is the Internet.

7. The method of claim 1 wherein the annotator accesses the annotation management engine through a web browser.

8. The method of claim 1 wherein the annotation management engine further comprises an encryption controller.

9. The method of claim 1 further comprising the step of creating one or more hotkeys corresponding to one or more labels and applying the corresponding label when the hotkey is selected by the annotator.

10. The method of claim 9 wherein the one or more hotkeys are shared over a network.

11. The method of claim 1 further comprising the step of using the checked data file to train a machine learning algorithm.

12. The method of claim 1 further comprising repeating the method until the body of data includes a checked data file for each of the plurality of data files of the body of data.

9058317 June 2015 Gardner
20180173886 June 2018 Dryer
20220230095 July 2022 Stergioudis
2014274590 January 2015

Bontcheva, GATE Teamware: a web-based, collaborative text annotation framework, pp. 1007-1029 (Year: 2013). cited by examiner
Islamaj, TeamTat: a collaborative text annotation tool, pp. 5-11, May 2020. cited by examiner
Yang, Yedda: A Lightweight Collaborative Text Span Annotation Tool, pp. 1-6 (Year: 2018). cited by examiner
Biemann, Collaborative Web-based Tools for Multi-layer Text Annotation, pp. 1-28, Jun. 2017. cited by examiner
Kehoe, eMargin: A collaborative text annotation tool, p. 263, Jun. 2012. cited by examiner

edspgr.12013841